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TITLE OF THE INVENTION 

Neural Network Processing System Using Semicon- 
ductor Memories 

BACKGROUND OF THE INVENTION 

The present invention relates to a data process- 
ing system having a memory packaged therein for real- 
izing a large-scale and fast parallel distributed pro- 
cessing and, more specifically, to a neural network 
processing system. 

The parallel distributed data processing using 
the neural network called the "neuro-compu t i ng** (as 
will be shortly referred to as the "neural network 
processing") is noted in the field of acoustics, 
speech and image processing, as described in either on 
pp. 145 - 168, "Parallel networks that learn to 
pronounce English text. Complex Systems 1 by Sej- 
nowski , T. J., and Rosenberg, C. R. 1987, or "Neural 
Network Process i ng" pub 1 i shed by Sangyo Tosho and 
edited by Hideki Asou. In the neural network process- 
ing, a number of processing elements called the "neu- 
rons" connected in a network exchange the data through 
transfer lines called the "connections" for high-grade 
data processing. In each neuron, the data (i.e., the 
outputs of the neurons) sent from another neuron are 
subjected to simple processing such as multiplications 
or summations. Since the processing in the individual 
neurons and the processing of different neurons can be 
carried out in parallel, the neural network processing 
is advantageous in principle in its fast data process- 
ing. Since algorithms (or learnings) for setting the 
connection weights of the neurons for a desired data 



processing have been proposed, the data processing can 
be varied for the objects, as described in either pp. 
533 - 536, "Learning representations by back-propaga- 
tion errors,". Nature 323-9 {1986a) by Rumelhart, D. 
5 E. , Hinton, G. E. and Williams, R. J., or in 2nd Sec- 
tion of "Neural Network Processing" published by San- 
gyo Tosho and edited by Hideki Asou, 

SUMMARY OF THE INVENTION 

10 First of all, the operating principle of the neu- 

ral network will be described in connection with two 
representative kinds: the mul t i - 1 ayered network and 
the Hopfield network. Fig. 2(a) shows the structure 
of the mul t i - 1 ayered network, and Fig. 3(a) shows the 

15 structure of the Hopfield network. Both of these net- 
works are constructed of the connections of neurons. 
Here are used the terminology of "neurons", which will 
be called the "nodes" or "processing elements", as the 
case may be. The directions of the connection arrows 

20 indicate those of transferring the outputs of neurons. 
In the mul t i - 1 ayered network, as shown in Fig. 2(a), 
the neurons are stacked in multiple layers so that the 
neuron outputs are transmitted in the direction from 
the input to output layers only. Input signals INi, - 

25 - and INn are inputted to the input layer, and out- 
put signals OUTx , - - and OUTn are outputted from 
the output layer. In the Hopfield network, on the 
other hand, the neuron outputs are fed back to an 
identical neuron and are transferred in two ways be- 

30 tween arbitrary two neurons. The feedback to the 
identical neuron may be omitted. 

Figs. 2(b) and Fig. 3(b) show the processing 
principle to be accomplished in the neurons. This 
processing principle is similar in any network and 

35 will be described In connection with the mu 1 t i - 1 ayered 
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network with reference to Fig- 2(b). Fig. 2(b) shows 
a j-th neuron in the (S+l)th layer in an enlarged 
scale. This neuron is fed through the connection with 
the output values Vxs, - - V^s, - - and Vnss of 
5 the neurons in the preceding layer, i.e., the S-th 

layer. Here, letters ns indicate the number of neu- 
rons in the S-th layer. In the neuron, the products 
VisT^ji, - - -, ViaT^ji, - - and V„ssT^j„a of the 
inputted output values Vis, - - Vis, - - and 

10 V„ss of the neurons and the connection weights T®ji 

and so on are calculated by means of a multiplier MT. 
Next, the sum of these products and an offset 0 js^i 
-is calculated by means of an adder ADD. The offset 0 
js-i may be omitted, as the case may be. Moreover, 

15 the result is inputted to a circuit D for nonlinear 

transfer function to obtain the output value Vjs^i of 
the neurons. The nonlinear transfer function circuit 
D has characteristics, as shown in Figs. 2(c) or 2(d), 
and outputs an output g(x) for an input x. Fig. 2(c) 

20 shows an example of the nonlinear transfer function 

for outputting a binary output gl or g2 in dependence 
upon whether or not the input x exceeds a predeter- 
mined threshold value xth, and Fig. 2(d) shows an ex- 
ample using a sigmoid function for issuing continuous 

25 outputs. The nonlinear transfer function circuit D is 
given other characteristics, if necessary. As the 
case may be, on the other hand, the circuit D may be 
given linear characteristics. 

The processing principle described above is also 

30 similar in the Hopfield network, as shown in Fig. 

3(b). In the Hopfield network, however, not only the 
output of the neuron of the layer preceding by one but 
also the outputs of all neurons are inputted to one 
neuron. In the mul t i - 1 ayered network, as seen from 

35 Figs. 2(a) and 2(b), one processing is ended by feed- 



i ng the output values of the neurons of the input 
layer at first and by updating the output values of 
the neurons in the next layer and then by the outputs 
values of the neurons of the secondary next layer. In 
5 the Hopfield network of Fig* 3(a), on the other hand, 
the output values of the individual neurons can be up- 
dated at suitable timings because of lack of any lay- 
er. In this Hopfield network, all the neuron output 
values are suitably given, and their updating is con- 

10 tinued till they come to an equilibrium state* The 
net work, in which the output values of all neurons 
are simultaneously updated, is called the "synchroniz- 
ed- Hopfield network", whereas the network, in which 
the output values are updated at arbitrary timings, 

15 are called the "unsynchron i zed Hopfield network" so 
that they are distinguished. 

One method used for accomplishing the aforemen- 
tioned neural networks has employed the software 
whereas the other the hardware. According to the 

20 method employing the software, the processing of 

neurons is carried out with a program written in com- 
puter languages so that the number or structure of the 
neurons can be easily changed* Since, however, the 
processing is sequentially performed, the former 

25 method is disadvantageous in that the data processing 
time is abruptly elongated for an increased number of 
neurons. In the Hopfield network us i ng an n number of 
neurons, an n times of products have to be calculated 
for updating the output of one neuron. In order to 

30 update the output values of all neurons at least once, 
therefore, an n^ times of products need to be calcu- 
lated* In other words, the number of calculations 
will increase in the order of n* with the increase in 
the neuron number n* As a result, the data processing 

35 time will increase in the order of n* if the multi- 



% -s- 90 



plications are sequentially accomp 1 i shed ♦ 

According to the method employing the hardware, 
the processing time can be shortened by changing the 
neurons to be multiplied into the hardware* Another 
5 trial for speeding up the processing has been made by 
executing the processing in parallel with a number of 
hardware neurons. If, however, the number of neurons 
is enlarged, the number of wiring lines acting as the 
signal lines between the neurons will increase in the 
10 order of n*, thus making it difficult to realize a 
1 arge-scal e ne twork. 

The method of solving the wiring problem is ex- 
emplified on pp-;— 123 - 129 of Nikkei Microdevice, 
March, 1989, as will be described in principle in Fig. 
15 4. Fig. 4 shows an example, in which a mul t i - 1 ayered 
network composedi of three layers each having three 
neurons is constructed of analog neuro-processors ANP 
and SRAM. The ANP is made by integrating one multi- 
plier Mt and one adder ADD of Fig. 2(b) and a non- 
20 linear transfer function circuit D into one chip. 

Another chip SRAM is stored with the connection weight 
belonging to each neuron. The neurons of different 
layers are connected through one signal line called 
the "analog common bus". Since the neuron output 
25 value of an input layer is inputted from the outside, 
the ANP and SRAM corresponding to the neurons of the 
input layer are omitted. 

The operations are as follows. First of all, the 
connection weight for each ANP necessary for a desired 
30 data processing is read in each SRAM from the outside. 
Next, an input analog signal corresponding to the out- 
put value of one neuron in the input layer is input- 
ted. The input signals are inputted in parallel di- 
rectly to the ANPs corresponding to the neurons of the 
35 middle layer. The weight data are read out from the 



SRAM to each ANP in synchronism with the input signal. 
Next, the product of two signals is calculated, and 
the result is stored in each ANP. Next, the input 
analog s ignals .correspondi ng to the output values of 
5 other neurons of the input layer are inputted, and 

their product is calculated. The calculation result 
is added to the value which is stored In each ANP of 
the middle layer. After similar calculations have 
been accomplished for the input analog signal cor- 
10 responding to the output value of the last neuron of 

the input layer, the neuron output values Vxa, V^a and 
Vaa of the middle layer are consecutively outputted to 
the "anal og "common bus of ' the' middl e 1 ayer by the non- 
^ linear transf er f unc t i on circuit in the ANP so that 

,;i5 calculations similar the foregoing ones are continued. 
v3 Finally, the neuron output values Vxa, V^a and V33 of 

the output layer are consecutively outputted to the 
m analog common bus of the output layer by the nonlinear 

= transfer function circuit in the ANP of the output 

iy20 layer. 

fU According to the example of the prior art de- 

it scribed above with reference to Fig. 4, the wiring 

C3 problem can be avoided by driving the common bus in 

the time sharing manner. Since, moreover, the multi- 
25 plications of the number of the neurons in one layer 
can be carried out in parallel, the data processing 
rate can be drastically raised, as compared with the 
method according to the software, as a result of the 
speed-up of the processing by the hardware. 
30 Since, however, the ANPs and the SRAMs are formed 

in different chips, it is difficult to realize the 
large-scale network in high density. Even if thirty 
two neurons could be integrated over a square chip of 
10 mm, as described on pp. 129 of the above-specified 
35 NikKei Micro Device, March, 1989, one adder, one mul- 
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tiplier and one nonlinear transfer function circuit 
need be prepared for each neuron. Hence, it is dif- 
ficult to integrate several hundreds or thousands neu- 
rons over one chip. 
5 Moreover, the examples of the prior art described 

above is accompanied by the following problems when 
they are applied to practices. An appl i cat i on of the 
multi-layered network is exemplified by outputting the 
pronunciations and accents of English letters inputted 

10 to a three- 1 ayered network, as described on pp. 145 - 
168 of Complex Systems 1 (1987) entitled "Parallel 
networks that learn to pronounce English text" and 
edited by Sejnowski, T. J., and-Rosenberg, C. R. Sev- 
en English letters are encoded as the neuron output 

15 values of the first layer, and the codes corresponding 
to the pronunciations and accents of the central one 
of the aforementioned seven letters are outputted as 
the output values of the twenty six neurons of the 
third layer. In this example, in dependence upon the 

20 inputs, the output values of the neurons of the output 
layer may fail to coincide with the codes of the pre- 
defined pronunciations and accents but may give fuzzy 
values. It is, therefore, necessary, to find out the 
codes closest to those of al 1 the pronunciations and 

25 accents compared and to make then as the correct an- 
swer. These comparisons between the output values and 
the desired values (i.e., the codes of the pronuncia- 
tions and accents in the aforementioned example) of 
the outputs are likewise required for the pattern 

30 recognition by the neural network. This point is not 
taken into consideration in the aforementioned ex- 
amples of the prior art so that the examples are in- 
convenient when applied to the practical problem. 

In the aforementioned examples of the prior art, 

35 moreover, the connection weight necessary for the 
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desired data processing is determined by an external 
computer, and the result is written in the SRAMs of 
Fig. 4. As a result, the learning is wholly performed 
by the software so that it is difficult to carry out 
at a high speed. 

In order to solve the problems thus far describ- 
ed, according to the present invention, there is pro- 
vided a neural network processing system which com- 
prises: a memory for storing neuron output values, 
connection weights, the desired values of outputs, and 
data necessary for learning; an input/output circuit 
for writing or reading data in or out of said memory; 
an ari thmetic""circui t for per forming- a processing for 
determining the neuron outputs such as the product, 
sum and nonlinear conversion of the data stored in 
said memory, a comparison of the output value and its 
desired value, and a processing necessary for 
learning; and a control circuit for controlling the 
operations of said memory, said input/output circuit 
and said arithmetic circuit. The arithmetic circuit 
is constructed to include at least one of an adder, a 
multiplier, a nonlinear transfer function circuit and 
a comparator so that at least a portion of the proces- 
sing necessary for determining the neuron output 
values such as the product or sum may be accomplished 
in parallel. Moreover, these circuits are shared 
among a plurality of neurons and are operated in a 
time sharing manner to determine the plural neuron 
output values. Still moreover, the aforementioned 
comparator compares the neuron output value determined 
and the desired value of the output In parallel. 

Since the adder, the multiplier and the nonlinear 
transfer function c i rcui t are shared among the plural 
neurons, as described above, the system for calculat- 
ing the neuron output values in the neural network 




using the numerous neurons can be realized by using a 
small number of circuits- Since, moreover, at least a 
portion of the neuron processing such as the product 
and sum is performed in parallel by the aforementioned 
5 arithmetic circuit, the data processing can be ac- 
complished at a high speed. Since, still moreover, 
the output value obtained and the desired value of the 
output can be compared in parallel by the aforemen- 
tioned comparator, the distance (i.e., the similarity 

10 between the obtained output value and the expected 
value, e.g., the humming distance) between the ob- 
tained output value and the desired value can be 
"-determined" at the high speed. - Si nee , furthermore, at 
least a portion of the processing necessary for the 

15 learning is accomplished by the hardware of the sys- 
tem, the learning can be speeded up. 

An object of the present invention is to solve 
the problems owned by the aforementioned prior art. 
Another object of the present invention is to 

20 provide a system for carrying out the calculations of 
neuron output values at a high speed with, a small num- 
ber of circuits in the network containing a number of 
neurons. 

Still another object of the present invention is 
25 to give the aforementioned system a function to com- 
pare the neuron output value and the desired value at 
a high speed. 

A further object' of the present invention is to 
give the aforementioned system a function to process 
30 at least a portion of the processing necessary for the 
1 earn i ng. 

These and other objects and many of the attendant 
advantages of the present invention will be readily 
appreciated as the same becomes better understood by 
35 reference to the following detailed description when 



considered in connection with the accompanying draw- 
ings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows one embodiment of the structure, in 
which the data processing system according to the 
present invention is embodied over a single semicon- 
ductor chip; 

Figs- 2(a) and 2(b) are diagrams showing the 
principle of the mul t i - 1 ayered neural network; 

Figs, 2(c) and 2(d) are diagrams showing the ex- 
amples of the characteristics of the nonlinear trans- 
fer-function circuit D; 

Figs. 3(a) and 3(b) are diagrams showing the 
principle of the Hopfield neural network; 

Fig. 4 shows an example of the neural network 
processing system using the plural chips according to 
the prior art; 

Figs. 5(a) and 5(b) show embodiments of the 
method for parallel processing of the mul t i - 1 ayered 
neural network and the Hopfield neural network, re- 
spectively; 

Figs. 6(a) and 6(b) show second embodiments of 
the method for parallel processing of the multi- 
layered neural network and the Hopfield neural net- 
work, respectively; 

Fig. 7(a) shows one embodiment of the structure, 
in which the data processing system according to the 
present invention is realized by using a memory array 
capable of reading out a number of data on data lines 
by selecting one word line; 

Fig. 7(b) shows one embodiment of the structure, 
in which the data processing system according to the 
present invention is realized by using two memory ar- 
rays capable of reading out a number of data on data 




lines by selecting one word line; 

Fig., 8(a) shows one embodiment of the correspond- 
ing relations of memory cells to the neuron output 
values and the connection weights, in case the multi- 
5 layered neural network is realized by using the em- 
bodiment of Fig. 7(b), that is to say, the embodiment 
in which the neuron output value and the connection 
weight are realized with each memory cell; 

Fig. 8(b) shows one embodiment of the character- 

10 istics of the nonlinear transfer function circuit D 

which is suitable in case binary memory cells are used 
in the embodiment of Fig. 8(a); 
Fig, 8 (c) "shows "one" embodiment- of the charac- 
teristics of the nonlinear transfer function circuit D 

15 which is suitable in case tetral memory cells are used 
in the embodiment of Fig. 8(a); 

Fig. 8(d) shows one embodiment of the method of 
selecting the word lines and the memory cells in the 
arithmetic mode in the embodiment of Fig. 8(a); 

20 Fig. 9(a) shows a second embodiment of the cor- 

responding relations of the memory cells to the neuron 
output values and the connection weights, in case the 
mul t i -1 ayered neural network is realized by using the 
embodiment of Fig. 7(b), that is to say, the embodi- 

25 ment in which the neuron output values and the connec- 
tion weights are realized by a plurality of memory 
cells; 

Fig. 9(b) shows one embodiment of the charac- 
teristics of the nonlinear transfer function circuit D 
30 which is suitable in case the neuron output values and 
the connection weights are equivalently expressed by a 
plurality of memory cells in the embodiment of Fig. 
9(a); 

Fig. 9(c) shows one embodiment of a method of 
35 setting the values Xthl, - - and Xthp of Fig. 9(b) 



for giving the nonlinear transfer function circuit D 
the sigmoid characteristics; 

Fig. 10(a) shows a third embodiment of the cor- 
responding relations of the memory cells to the neuron 
5 output values and the connection weights, in case the 
mul t i - 1 ayered neural network is realized by using the 
embodiment of Fig. 7(b), that is to say, the embodi- 
ment in which the neuron output values and the connec- 
tion weights are realized by a plurality of memory 
10 cells; 

Fig. lp(b) shows one embodiment of the charac- 
teristics of the nonlinear transfer function circuit D 
C3 which is suitable in case the neuron output values and 

JtJ the connection weights are binarily expressed by a 

IJ15 plurality of memory cells in the embodiment of Fig. 
10(a); 

m Fig. 11 shows one embodiment in case a DRAM cell 

is used as the memory cell of the embodiment of^Fig. 
U 8(a); 

rU20 Fig. 12 shows one embodiment of the relations be- 

[.^ tween the arithmetic modes and the external signals; 

□ Fig. 13(a) shows one embodiment of the operation 

waveforms in the memory mode of the embodiment of Fig. 

11; 

25 Fig. 13(b) shows one embodiment of the operation 

waveforms in the arithmetic mode of the embodiment of 
Fig. 11; 

Fig. 14(a) shows' one embodiment of the nonlinear 
transfer function circuit D which is suitable for the 
30 embodiment of Fig* 11 or the like; 

Fig. 14(b) shows one embodiment of the charac- 
teristics of the nonlinear transfer function circuit D 
of Fig. 14(a) ; 

Fig. 14(c) shows one embodiment of the input/out- 
35 put circuit 10 which suitable for the embodiment of 
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Fig- 11 or the like; 

Fig* 14(d) shows one embodiment of the potential 
relation between the product and sum output line NO 
and the write line lA in case the nonlinear transfer 
5 function circuit of Fig. 14(a) and the write circuit 
of Fig. 14(c) are used; 

Fig. 14(e) shows one embodiment of the read cir- 
cuit OUT which is suitable for the embodiment of Fig. 
11 or the like; 

10 Fig. 14(f) shows one embodiment of the read latch 

circuit OUTLT which is suitable for the embodiment of 
the read circuit OUT of Fig. 14(e); 

Fig. 15 shows" a' second ' embo'drment of the- multi- 
plier MT in Fig. 11 ; 

15 Figs. 16(a) and 16(b) show examples of the SRAM 

eel 1 ; 

Fig. 16(c) shows one embodiment in case the SRAM 
cells of Figs, 16(a) and 16(b) are used as the memory 
cell of Fig, 8(a) ; 
20 Fig, 17(a) shows one embodiment of the operation 

waveforms in the memory mode of the embodiment of Fig. 
16(c) ; 

Fig. 17(b) shows one embodiment of the operation 
waveforms in the arithmetic mode of the embodiment of 
25 Fig. 16(c); 

Fig. 18(a) shows one embodiment in case the DRAM 
cells are used in the embodiment of Fig. 9(a) or the 
embodiment of Fig. 10(a); 

Fig. 18(b) shows one embodiment of the structure 
30 of the block BLKl in the embodiment of Fig. 18(a); 

Fig. 18(c) shows one embodiment of the structure 
of the block BLK2 and the nonlinear transfer function 
circuit D in the embodiment of Fig. 18(a); 

Fig. 18(d) shows one embodiment of the structure 
35 of the individual nonlinear transfer function circuits 



DSx (X = 1, 2, - - -» and p) composing the nonlinear 
transfer function circuit DIO in the embodiment of 
Fig. 18(c); 

Fig. 18(e) shows one embodiment of the charac- 
5 teristics of the individual nonlinear transfer func- 
tion circuit DSx (x = 1, 2, - - and p) composing 
the nonlinear transfer function circuit DIO in the em- 
bodiment of Fig- 18(c); 

Fig, 19(a) shows one embodiment of the structure 
10 of the nonlinear transfer function circuit DIO which 
is suitable in case the neuron output values and the 
connection weights are binarily expressed in a plu- 
Q rality of memory cells in the embodiment of Fig, 

jD 18(a); 

ij.5 Fig, 19(b) shows one embodiment of the charac- 

\0 teristics of the nonlinear transfer function circuit 

1^ DSx (X = 1, 2, - - and z) in the embodiment of Fig. 

CO 18(a); 

Fig. 19(c) shows one embodiment of the charac- 
m20 teristics in the embodiment of Fig. 19(a); 

Fig. 19(d) shows one embodiment of the structure 
Q of the encoder in the embodiment of Fig. 19(a); 

53 Fig. 20(a) shows one embodiment of the corres- 

ponding relations of memory cells to the neuron output 
25 values and the connection weights, in case the un- 
synchronized Hopfield neural network is realized by 
using the embodiment of Fig. 7(b), that is to say, the 
embodiment in which the neuron output value and the 
connection weight are realized with each memory cell; 
30 Fig. 20(b) shows one embodiment of the corres- 

ponding relations of memory cells to the neuron output 
values and the connection weights, in case the syn- 
chronized Hopfield neural network is realized by using 
the embodiment of Fig. 7(b), that is to say, the em- 
35 bodiment in which the neuron output value and the con- 



nection weight are realized with each memory cell; 

Fig. 21(a) shows one embodiment of the corres- 
ponding relations of memory cells to the neuron output 
values and the connection weights, in case the unsyn- 
chronized Hopfield neural network is realized by using 
the embodiment of Fig. 7(b), that is to say, the em- 
bodiment in which the neuron output value and the con- 
nection weight are realized with a plurality of memory 
eel 1 s ; 

Fig- 21(b) shows one embodiment of the corres- 
ponding relations of memory cells to the neuron output 
values and the connection weights, in case the syn- 
chronized Hbpfreld~neurar network is realized by using 
the embodiment of Fig- 7(b), that is to say, the em- 
bodiment in which the neuron output value and the con- 
nection weight are realized with a plurality of memory 
cells; 

Fig- 22 shows one embodiment in case the neuron 
output values and the connection weights are enabled 
to take positive and negative values by using coding 
bits; 

Fig. 23 shows one embodiment in case the system 
according to the present invention is given a function 
to compare the neuron output values and the desired 
values; 

Fig. 24 shows one embodiment of the comparator 
for comparing the data read out to a plurality of data 
line pairs of the memory cell array TG and the memory 
cell array A to calculate the extent of similarity of 
the data; and 

Fig- 25 shows one embodiment in which the updat- 
ing of the neuron output values is speeded up by pro- 
viding a register. 



DETAILED DESCRIPTION OF THE INVENTION 



Fi^, 1 shows one embodiment in case the data 
processing system according to the present invention 
is integrated over a semiconductor chip« 

There are integrated over a semiconductor chip 
5 (CHIP): a memory (A) for storing data; an input/output 
circuit (I/O) for performing at least one of the writ- 
ing operation and reading the data in and from said 
memory; an arithmetic circuit (ARTM) for performing 
the ar i thmet i c f or determining neuron output values, 

10 the comparison (i.e., the similarity of the obtained 
output values and the desired values, e»g., the cal- 
culation of the humming distance) of the output values 
and the desired values or the" ari thmet i c necessary for 
the learning by using the data stored in said memory; 

15 and a control circuit (CNT) for controlling the opera- 
tions of said memory, said input/output circuit and 
said arithmetic circuit. Each of buses (BUSl, BUS2, 
BUSS, BUS4, etc.) connecting the individual blocks is 
made of not only one wiring line but also a necessary 

20 number of wiring lines. The aforementioned memory can 
be stored with the connection weights and the neuron 
output values necessary for the neural network proces- 
sing, the desired values of the outputs or the data 
necessary for the learning. According to the present 

25 embodiment, the nonlinear network processing such as 

the calculations of the neuron output values, the com- 
parisons of the desired values and the desired values, 
or the calculations necessary for the learning can be 
performed in the following manners. 

30 First of all, the method of calculating the neu- 

ron output values will be described in the following. 
At first, the connection weights necessary for the 
calculations for the neural network processing, and 
the neuron output values, or the offsets are read out 

35 in parallel from the memory to the arithmetic circuit 




throu;Ech a bus 1. Next, the arithmetics such as the 
product and sum or the nonlinear transfer necessary 
for determining the neuron output values are accom- 
plished by the arithmetic circuit, and the obtained 
5 results are written In the memory through the Input/ 
output circuit. The operations described above are 
continued by a necessary number of times to determine 
the neuron output values* The arithmetic circuit may 
either determine one of plural neuron output values by 
10 a single operation or perform a portion of calcula- 
tions for determining the neuron output values. Thus, 
the data processing can be accomplished by the various 
networks'such as~"the multi-layered n 

synchronized or unsynchron i zed Hopfield network. In- 

15 cidentally, in order to update the cTutput values of 
all the neurons synchronously, the synchronized Hop- 
field network needs to be stored with the output 
values of all the neurons till the end of updating the 
output values of all the neurons- In this case, the 

20 output values of all the previous neurons may be 

stored in the memory so that they may be used for up- 
dating the output values of the neurons. 

According to the present embodiment, a desired 
number of multipliers, adders and nonlinear transfer 

25 function circuits necessary for calculating the neuron 
output values may be provided in the arithmetic cir- 
cuit so that they may be repeatedly used* This makes 
it possible to make tlie number of circuits far smaller 
than that of the case in which those circuits are 

30 prepared for each of the neurons. The example of the 
prior art of Fig- 4 is required to prepare two hund- 
reds multipliers, adders and nonlinear transfer func- 
tion circuits for realizing the mul t i - 1 ayered network 
having three layers each composed of one hundred neu- 

35 rons. In the present embodiment, on the contrary, it 



is sufficient to prepare at least one multiplier, at 
least one adder . and at least one nonlinear transfer 
function circuit. Even if the multiplications neces-' 
sary for updating one neuron output value to speed up 
5 the operations should be accomplished in parallel, it 
would be sufficient to prepare one hundred multi- 
pliers, one adder and one nonlinear transfer function 
circuit. According to the present embodiment, there- 
fore, the number of circuits can be drastically re- 

10 duced, as compared with that of the prior art. In- 
cidentally, the above-specified different will become 
the larger for the larger scale of the network. Simi- 
lar s i tuat i ons will apply to another network such as 
the Hopfield network. 

15 Not only the arithmetic speed such as multiplica- 

tions but also the amount of arithmetics to be carried 
out in parallel makes high contribution to the data 
processing speed in case the neuron output values are 
to be determined. In the Hopfield network using an n 

20 number of neurons, for example, the products of n* 
times have to be calculated for updating the output 
values of all the neurons, as has been described 
hereinbefore. If the multiplications are sequentially 
accomplished, therefore, the updating of the output 

25 values of all the neurons takes at least a time period 
of n* times as long as that required for one multi- 
plication. As a result, the time period required for 
the multiplications will abruptly increase in the or- 
der of square of the neuron number with the increase 

30 in the number of neurons. Similar circumstances will 
also apply to the mul t i - 1 ayered network. This makes 
it desirable to calculate the numerous multiplications 
in parallel. Here will be described in the following 
an example of the arithmetic system for raising the 

35 data processing speed by making the multiplications in 




parallel so as to determine the neuron output values 
in the embodiment of Fig, 1* 

Fig* 5 Illustrates the mul t i - 1 ayered network at 
(a) and the Hopfield network at (b) of one system for 
5 the parallel arithmetics. In the present embodiment, 
the products necessary for determining one neuron out- 
put value are calculated, as shown. Specifically, the 
output value of the neuron of the preceding layer, 
which is inputted to one neuron, and the connection 

10 weight for said output value of the neuron under con- 
sideration are read out in parallel from the memory, 
and their products are calculated in parallel. Thus, 
the time per i od requi red f or the- mul t i p 1 i ca t i ons will 
increase in the order of the neuron number with the 

15 increase in the neuron number. As a result, the data 
processing time can be drastically shortened, as com- 
pared with the case in which the multiplications are 
sequentially accomplished. In Fig. 5, only the multi- 
plications necessary for updating the output value of 

20 one neuron are accomplished in parallel. However, the 
embodiment of Fig. 1 should not be limited thereto but 
may naturally add the arithmetic circuits within a 
range allowed by the degree of integration, to update 
the output values of the plural neurons in parallel. 

25 In this case, the data processing can be accomplished 
at a higher speed. In addition, the parallel arith- 
metics can also be accomplished by another system, as 
shown in Figs. 6(a) and 6(b). 

Fig. 6 shows one embodiment, in which the multi- 

30 plications are executed in parallel for a plurality of 
neurons to be fed with the output value of one neuron 
in the mul t i - 1 ayered network of Fig. 6(a) and in the 
Hopfield' network of Fig* 6(b). In this method, the 
neural output values and the connection weights are 

35 read out from the memory, and the calculations neces- 




sary for updating the neuron output values are ex- 
ecuted bit by bit for the plural neurons. This makes 
it impossible to realize the unsynchron i zed Hopfield 
network- Since, however, the time period required for 
5 the multiplications will increase in the order of the 
neuron number with the increase in the neuron number 
like the system of Fig. 5, the data processing time 
can be drastically shortened, as compared with the 
case in which the multiplications are sequentially 

10 carried out. 

In the example of the prior art of Fig. 4, too, 
the arithmetics are executed in parallel. As will be 
de"scribed in the fo 1 rowi ng, however , the structure of 
Fig. 1 can be realized with a smaller number of cir- 

15 cuits than that of the example of the prior art. In 
the systems shown in Figs. 6(a) and 6(b), only one 
mul t i p 1 i er operates in parallel in each neuron, as 
hatched. In the embodiment of Fig. 1, therefore, the 
arithmetic circuit may be provided with the multi- 

20 pliers in a number equal to that of the neurons to be 
calculated at one time so that this system can be 
realized with a smaller number of circuits than that 
of the case of the prior art in which all the multi- 
pliers are provided for all the individual neurons. 

25 In the mul t i - 1 ayered network having three layers each 
composed of three neurons, for example, a similar par- 
allelism can be realized by using individually three 
multipliers, adders and nonlinear transfer function 
circuits, for example, according to the embodiment of 

30 Fig. 1, on the contrary to the case of the prior art 
which is equipped with individually six multipliers, 
adders and nonlinear transfer function circuits. 

Thus, according to the embodiment shown in Fig. 
1, a system for the data processing similar to that of 

35 the neural network using numerous neurons can be real- 
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ized with the necessary minimum circuits by sharing 
the adders, multipliers and nonlinear transfer func- 
tion circuits of the arithmetic circuit among the 
plural neurons. By executing the arithmetics such as 
5 the products or sums with the aforementioned arith- 
metic circuit, moreover, the data processing can be 
accomplished at a high speed. Incidentally, with the 
parallel arithmetics, the number of wiring lines be- 
tween the memory and the arithmetic circuit has to be 

10 increased to send many data at once to the arithmetic 
circuit. In Fig. 1, however, the memories and the 
arithmetic devices are arranged over the common chip 
so that the number _.of.„the wi ring J i nes of the bus can 
be easily increased, 

15 Although the method of calculating the neuron 

output values has been described hereinbefore, a neu- 
ron output value and its desired value can be compared 
according to the embodiment of Fig, 1, For this com- 
parison, the desired value may be stored in advance in 

20 the memory so that its distance from the output value 
obtained by the aforementioned method may be calcu- 
lated by the arithmetic circuit. This operation is to 
calculate the similarity between the desired value and 
the calculated value. At this time, the desired value 

25 and the output value composed of numerous bits can be 
simultaneously read out to the arithmetic circuit and 
processed in parallel with ease by increasing the num- 
ber of wiring lines of the bus 1, Thus, according to 
the embodiment of Fig, 1, the data processing such as 

30 the pattern recognition can be executed at a high 
speed, as compared with the case in which the com- 
parison is accomplished serially bit by bit by using 
an external computer. 

According to the embodiment of Fig, 1, moreover, 

35 the learning can be accomplished at a higher speed 



than that of the case using the software, by executing 
the arithmetics necessary for the learning with the 
arithmetic circuit. This specific embodiment will be 
described hereinafter* 
5 The neural network processing is advantageous in 

that it can process various data by changing the con- 
nection weights* This advantage can be easily ex- 
ploited according to the embodiment of Fig. 1 by re- 
writing the connection weight stored in the memory. 

10 Moreover, several kinds of connection weights neces- 
sary for different data processings can be stored in 
advance by making the capacity of the memory larger 
than that necessary" for calculating the neuron output 
values. In this case, there can be attained a merit 

15 that different kinds of data can be continuously 

processed without losing the time period for rewriting 
the connection weights. In addition, in. case numeral 
input data are to be continuously processed, the nec- 
essary input data or the obtained data can be stored 

20 in advance in a portion of the memory. Thus, the 

frequency for switching the reading, calculating and 
outputting modes can be reduced to shorten the pro- 
cessing time, as compared with the case in which the 
operations of reading each input data in the memory 

25 and calculating and outputting it to the outside of 
the system are repeated. 

Next, a more specific embodiment based upon the 
embodiment of Fig. 1 will be described in the follow- 
ing. For simplicity of descriptions, the case of 

30 giving the arithmetic circuit the function of cal- 
culating the neuron output values will be described at 
first, and the method of giving the comparing or 
learning function will be described at last. 

Fig. 7(a) shows one embodiment in case a lattice- 

35 shaped memory cell array is used in the memory of the 



embodiment of Fig. 1- In Fig. 7(a), letter A desig- 
nates a memory cell array which is composed of: a 
plurality of data lines (D); a plurality of word lines 
(W) arrayed to intersect the data lines (D) ; and memo- 
5 ry cells (MC) arrayed at the desired intersections- 

As a result, the signals of the different memory cells 
can be read out onto the plural. data lines by select- 
ing one of the word lines. Numeral 12 designates an 
arithmetic circuit (ARTM) • Numerals 10, 11, 13, 14, 

10 15 and 16 designate circuits corresponding to the con- 
trol circuit (CNT) of Fig. 1- The numerals 10 and 15 
designate address buffers for X-addresses and Y-ad- 
dr esses, and ^the numeral s 11 and 14 designate a de- 
coder and a driver for X-addresses and Y-addresses, 

15 respectively- The numeral 13 designates an array con- 
trol circuit for con tro 1 1 i ng the memory cell array. 
Numeral 16 designates a clock generator for generating 
clocks to controls the operations of the memories on 
the basis of the signals inputted from the outside. 

20 Letters OUT and WR designate a read circuit and write 
circuit, respectively. A chip select "CS is a chip 
selection signal. A write control signal WE is a sig- 
nal for switching the write and read operations for 
establishing the write operation at a low level and 

25 the read operation at a high level. Le t ter "NE des ig- 
nate an arithmetic circuit control signal for starting 
the arithmetic circuit at a low level and interrupting 
the same at a high level to act as an ordinary memory. 
In the following, the state of the signal "NE at the 

30 high level will be called the "memory mode", and the 
state at the low level will be called the "arithmetic 
mode". In the memory mode,' a desired memory cell is 
selected according to the X-address and the Y-address 
so that a write data DI can be written in that cell or 

35 so that a data can be read out from the same cell and 
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outputted as a read data DO. In the arithmetic mode, 
the data stored in the memory cell is read out to the 
arithmetic circuit 12 so that the arithmetic result of 
the arithmetic circuit 12 or the data according to the 
5 arithmetic result can be written in the memory cell 
through the input circuit. By selecting one word 
line, according to the present embodiment, the data of 
all the memory cells on the selected word are out- 
putted to the data lines. As a result, numerous data 

10 can be easily latched in the arithmetic circuit 12 so 
that many arithmetics can be accomplished in parallel. 
In order to calculate the neuron output values accord- 
"ingtb the present embodiment, the mode is changed at 
first into the memory mode to stop the arithmetic cir- 

15 cult, and the necessary connection weight, neuron out- 
put value (i.e,, the input signal at first), offset 
and so on are written in the memory. Next, the mode 
is changed to the arithmetic mode to start the arith- 
metic circuit to read the necessary data is read out 

20 to the arithmetic circuit by selecting one word line- 
Next, the result is written in the memory circuit. If 
the read of the data necessary for the arithmetics and 
the write of the result are further continued by a 
necessary number of times, the neural network process- 

25 i ng can be accomplished at a high speed. As has been 
described above, according to the embodiment shown in 
Fig. 7(a), many data can be written at once in the 
arithmetic circuit, the embodiment is suited for the 
parallel arithmetics of the type shown in Fig. 5 or 

30 Fig. 6. Thus, according to the present embodiment, 
the parallel arithmetics make it possible to execute 
the neural network processing at a high speed. By 
using the arithmetic circuit 12 repeatedly, moreover, 
the plural neurons can share the output value cal- 

35 culating circuit to effect a high integration easily. 
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In case, on the other hand, parallel arithmetics 
are to be accomplished by using the data stored in the 
memory cells on the plural word lines, a register for 
primary storage can be provided in the arithmetic cir- 
cuit so that it may once store the data obtained by 
selecting the word lines and execute the arithmetics 
of the stored data together with the data read out by 
selecting other word lines. 

As in the embodiment shown in Fig. 7(b), more- 
over, two memories A and B can be provided. In Fig. 
7(b). characters 13A and 13B designate array control 
circuits for controlling the memory cell arrays A and 
B, respectively. ~Other circuits such as a decoder is 
not shown in Fig. 7(b). According to the structure of 
Fig. 7(b), the data of the memory cells on the two 
word lines of the memory cell arrays A and B can be 
written in the arithmetic circuit by-selecting one 
word line of each of the memory cell arrays A and B. 
By using the structure of Fig. 7(b), the memory arrays 
can be separately used according to the kinds of data 
such that the memory cell array A is stored with the 
neuron output value whereas the memory cell array B is 
stored with the connection weight, the controls of the 
reading or writing operations can be simplified. In- 
cidentally, in the embodiments of Figs. 7(a) and 7(b), 
the write data DI and the read data DO may be process- 
ed in plurarity and in parallel, or the array A and B 
may be separately provided with the read circuit OUT 
and the write circuit WR. 

In the embodiments of Figs. 7(a) and 7(b). the 
selection of a specific memory cell can be accom- 
plished like the ordinary memory according to the 
address. By changing the order of selecting the 
address, therefore, those embodiments can be flexibly 
applied to the various networks or various parallel 



arithmetic systems. 

In the embodiments of Figs- 7(a) and 7(b), the 
memory can be exemplified by a highly integrated semi- 
conductor memory such as the DRAM or SRAM. In this 
5 case, the memory can store many data so that a large- 
scale network can be integrated into one chip. 

Next, the method of realizing the mul t i - 1 ayered 
network by using the structure of Fig. 7(b) will be 
described in detail in the following. The parallel 
10 arithmetic system is exemplified by taking the system 
of Fig, 5(a). It is assumed that the number of layers 
be m and that the number of neurons in each layer be 
n. Incidentally, the offset 0 of each neuron, as 
iO shown in Figs. 2(b) or Fig. 3(b), will be omitted here 

,ll5 so as to simplify the description. As is apparent 

from Fig. 2(b) or Fig. 3(b), however, the offset 6 of 
l2 each neuron can be handled like the output from an- 

IQ other ordinary neuron either by providing one neuron 

- having an output value of 1 at all times to setting 

Wp the connection weight of it and each neuron at the 
lU offset 0 or by increasing the neuron output value, 

which is to be inputted for each neuron, by 1 to set 
Q the value at the offset 0 of each neuron and the cor- 

responding connection weight at 1 so that their pro- 
25 duct may be added to the total sum of the products of 
other neuron output values and the connection weights. 
Fig. 8(a) shows one embodiment in which the memory 
cells are correspond io the connection weights and the 
neuron output values. Letter D designates the non- 
30 linear transfer function circuit; characters cl,* c2, - 
- -, and cn designate the adders; and characters ml, 
m2 , - - and mn designate the multipliers. The ad- 
ders cl, c2, - - cn cons t i tute al together the 
multi-input adder ADD of Fig. 2(b). The memory cell 
35 array A is stored with the neuron output values, and 
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the memory cell array B is stored with the connection 
weijBjhts. Although what is shown in Fig. 8(a) is the 
memory cells for storing the neuron output values and 
the connection weights, it is quite natural that the 
5 memory cells to be stored with other data such as the 
offsets 0 of the individual neurons or the data nec- 
essary for the learning may be provided, if necessary. 
As shown, the memory cells located at the intersec- 
tions of the word lines s and the data lines i in the 
10 memory cell array A are stored with neuron output 

values Vis, In other words, the output values of the 
neurons of the common layer are arranged on the common 
word line* In the memory cell array B, the memory 
cells located at the intersections between the word 
,""!l5 lines (s, j) and the data lines i are stored with con- 
in nection weights T^u* 

Figs, 8(b) and 8(c) show one embodiment of the 
rg input/output characteristics of the nonlinear transfer 

' function circuit D. Fig- 8(b) shows the embodiment 

n:20 having binary outputs gl and g2 • Characters xl and x2 
W indicate the lower limit and the upper limit of the 

;S input x, respectively. In Fig. 8(b), the output is 

C3 g2 , if the input x exceeds the threshold value xth, 

but gl if not. Therefore, the embodiment of Fig. 8(b) 
25 is suitable when the memory cell used is binary. Fig. 
8(c) shows an embodiment having tetral outputs ga and 
gb between the outputs gl and g2 . The present embodi- 
ment is a suitable example when the tetral memory 
cells are used. The gap between the elements gl , ga, 
30 and g2 can naturally be changed, if necessary, al- 

though they are shown equally distant in Fig. 8(c). 
In case the memory cells are exemplified by those 
capable of storing data having continuous values, 
i.e., the so-called "analog values", the nonlinear 
35 transfer function circuit D to be used may have the 



characteristics shown in Fig. 2(d). 

Fig, 8(d) shows one embodiment of the correspon- 
dences in the embodiment of Fig. 8(a) between the word 
line selecting method for determining the neuron out- 
put values of the final layer from the neuron output 
values of the input layer and the write destination 
addresses- The operations of Fig. 8(a) will be de- 
scribed in the following with reference to Fig. 8(d). 
The neuron output values Vn, Vax, - • and V„x of 
the input layer are written in advance in the input/ 
output circuit (although omitted from Fig. 8) in the 
memory cells on the word line of S = 1 of the array A. 
First of all, the word 1 i nes o f s =1 of the array A 
and the word lines of (s, j) = (1, 1) of the array B 
are selected simultaneously, although not necessarily 
complete. Then, the neuron output values Vn, Vax, - 

- and V„i of the input layer are outputted to the 
data lines of i = 1, 2, - - and n of the array A. 
On the other hand, the connection weights T^xxi T^x2» 

- - and T^xn are outputted to the data lines of i = 
li 2, - - and n of the array B. These values are 
inputted to the multipliers ml, m2 , - - and mn so 
that their products T^xxVxx, T^xaV^x, - - and 
T^xnVnx are inputted to the adders cl , c2, - - and 
cn. The results (T^xxVxx + T^iaVax, - - + T^xnV„x) 
are inputted to the nonlinear transfer function cir- 
cuit D. The output of thi s nonl i near transfer func- 
tion circuit D is written through the write circuit WR 
(although omitted) in the memory cells corresponding 
to the write destination addresses of (s, i) = (2, 1) 
in the array A. Thus, the value of the first neuron 
output value Vax of the second layer is calculated. 
Next, the word line of s = 1 of the array A and the 
word line of (s, j) = (1, 2) of the array B are simul- 
taneously selected. Then, the neuron output values 



-29- 



Vix, Vxi, - - and Vni of the input layer are out- 
putted to the data lines of i = 1, 2. - - and n of 
the array A. On the other hand, the connection 
weights T'-ai, T^aat - - and T^2„ are outputted to 
5 the data lines of i = 1, 2, - - and n of the array 
B. These values are inputted to the multipliers ml, 
m2, - - and mn so that their products T^aiVn, 
T^zaVar, - - and T^artVnx are inputted to the adders 
cl, c2, - - and cn . The results (T^axVxx + 
10 T^aaVax, " " + T^anVnx) are inputted to the non- 
linear transfer function circuit D. The output of 
this nonlinear transfer function circuit is written 
through the wr i te_c.i rcui t (although not shown) in the 
^5 memory cells corresponding to the write destination 

%a5 addresses of (s, i) = (2, 2) in the array A. Thus, 

the value of the second neuron output value V^a of the 
H second layer is calculated* All the neuron output 

values can be calculated by continuing the operations 
= thus far described according to Fig, 8(d). According 

itZO to the present embodiment, one neuron output value can 
m be determined by executing the reading and writing 

J3 operations once in the arithmetic mode so that the 

m neural network processing can be accomplished at a 

high speed. Since, moreover, the arithmetic circuit 
25 can be shared among all the neurons, a high integra- 
tion can be made. Incidentally, Fig. 8(d) shows one 
example of assignment of the memory cells, and the 
present invention should not be limited thereto but 
can be modified in various manners. For example, the 
30 plural input data can be continuously processes, as 
has been described hereinbefore. In this case, a 
plurality of sets of neuron output values of an input 
layer are required. For this operation, the neuron 
output values of the input layer corresponding to the 
35 plural input data may be written in advance on the 




plural different word lines of the array A so that 
they may be consecutively used. Thus, the neuron out- 
put values of the input layer need not. be read, in for 
each data processing so that the data processings can 
5 be continuously accomplished at the high speed* 

Here is used one memory cell for storing the 
neuron output value and the connection weight. This 
allows only binary values to be taken as the neuron 
output values and the connection weights in case the 
10 binary memory cells are used. By using the multi- 
valued memory cells, as has been described herein- 
before, the neuron output values and the connection 
weight values could be increased, but the multi-valued 
memory cells may have its reliability made deficient 
./is by the problem of S/N ratio. In this case, a plurali- 
ty of memory cells can be used for storing the neuron 
output values and the connection weights, as will be 
described in the following. 
- Fig. 9(a) shows one embodiment of the case in 

ij|0 which a p number of memory cells are used for storing 
fU one neuron output value and in which a q number of 

J5 memory cells are used for storing one connection 

p weight. The suffix i, j or s appearing in Fig. 9(a) 

to indicate the neuron output value or the connection 
25 weight corresponds to that of the embodiment shown in 
Fig. 8. In the embodiment of Fig. 9(a), the p number 
of continuous memory cells. on one word line in the ar- 
ray A express one neuron- output value, and the q num- 
ber of continuous memory cells on one word line in the 
30 array B express one connection weight. 

The calculations of the neuron output values are 
carried out in the following manner. First of all, 
like the embodiment of Fig. 8, the word line of s = 1 
of the array A and the word line of (s, j) = (1, 1) of 
35 the array B are simultaneously selected. Then, to the 



data line group of i = 1, 2, - - and n composed of 
p number data lines of the array A, there are out- 
putted the data expressing the neuron output values of 
Vxi, V^i, - - and Vtii of the input layer, which are 
5 inputted group by group to the adders al , a2 , - - 
and an. To the data line group of i = 1, 2, - - 
and n composed of q number of data lines of the array 
B, on the other hand, there are outputted the data ex- 
pressing the connection weights T^ix, T^ia, - - -, and 
10 T^m, which are inputted group by group to the adders 
bl, b2, - - and b3» By the aforementioned adders 
al , a2 , - - -i and an, and bl, b2, - - -, and bn, the 
neuron output values Vn, Y^i, - - and Vni and the 
In connection weights T^xx, T^ia, - - and T^in are 

composed and inputted, as shown, to the multipliers 
ml, m2 , - - and mn to produce the products T^ixVxi, 
T^xzVaxi - - -i and T^xnVnx. These products are in- 
putted to the adders cl, c2, - - -, and cn so that 
their results (T^xxVxx + T^x^V^x, - - + T^xnVnx) 
1%Q are inputted to the nonlinear transfer function cir- 
fU cui t D. The output of the nonlinear transfer function 

J~ circuit is written through the write circuit WR (al- 

P though not shown) to the p number of memory cell 

groups corresponding to the write destination address 
25 (s, i) = (2, 1) in the array A* The output values of 
all the neurons can be determined by continuing simi- 
lar operations by using the same address as that of 
Fig. 8(d). 

Since, according to the aforementioned embodi- 
30 ment, one neuron output value is expressed with the p 
number of continuous memory cells on one word line in 
the array A, the multi-valued neuron output values can 
be expressed by using the binary memory cells. Since, 
moreover, one connection weight is expressed with the 
35 q number of continuous memory cells on one word line 
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in the array B, the mul t i - val ued connection weights 
can be expressed by using the binary memory cells. As 
a result, the multi-valued such as the neuron output 
values or the connection wei;?hts can be expressed with 
the binary memory cells. In the aforementioned em- 
bodiment, moreover, the frequency of switching the ad- 
dresses is identical to that of the embodiment of Fig- 
8 so that the data can be processed at a high speed 
like the embodiment of Fig, 8. In order to write the 
result of the nonlinear transfer function circuit in 
the p number of memory cells expressing the neuron 
output values, the p number of writing operations may 
be -con tinuously "executed but can be easily accomplish- 
ed in parallel by providing the p number of write cir- 
cuits. In this case, it is possible to avoid the time 
losses for writing the plural memory cells. By pro- 
viding a plurality of read circuits, on the other 
hand, it is naturally possible to increase the speed 
of the reading operations. Incidentally, if a multi- 
input circuit is used as the multipliers ml, - - -, 
and mn in the embodiment of Fig, 9Ca), a similar cir- 
cuit can be realized without providing the adders al , 
- - and an, and bl, - - -, and bn. In addition, 
the structure of the arithmetic circuit can be modi- 
fied in various manners. 

The embodiment of Fig. 9(a) uses the p number of 
memory cells for storing the neuron output values and 
the q number of memory cells for storing the connec- 
tion weights. In other words, the neuron output 
values are expressed in the p bits, and the connection 
weights are expressed in the q bits. Since there are 
various methods of expressing the data in the plural 
bits, an expression method may be selected, if neces- 
sary, and it is accordingly sufficient to design the 
characteristics of the adders, the multipliers and the 
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nonlinear transfer function circuits. For example, 
the neuron output values can be expressed with the 
number of such ones of the memory cells of p bits ex- 
pressing the neuron output values as have a content of 
5 1. Fig. 9(b) shows one embodiment of the input/output 
characteristics of the nonlinear transfer function 
circuit D suited for such case. In Fig. 9(b), charac- 
ters gl , g2 , - - and gp designate the p number of 
outputs of the nonlinear transfer function circuit D. 
10 Their individual outputs take the value 0 or 1, which 
is written in the p number of corresponding memory 
cells through the write circuit WR (although not 

p shown). The outputs gl , g2 , - - and gp take the 

;S value 1, when the individual inputs exceed the values 

his xthl, xth2, - - and xthp, and otherwise take the 
C; value 0. These values xthl, xth2, - - -, and xthp may 

1^ be at an equal or arbitrary distance between the upper 

£0 limit xl and the lower limit x2 of the inputs. If the 

distance between the values xthk and xthk+1 (k = 1, - 
fiO - -, and p-1) is set, as shown in Fig. 9(c), the non- 
linear transfer function circuit g can be realized to 
n have the sigmoid characteristics. According to the 

53 present embodiment, one neuron output value can be 

given the p number of values by the p number of memory 
25 cells. In the present ^embodiment, the p number of 

memory cells expressing the neuron output values are 
equivalently handled. Specifically, the influences 
upon the neuron output' values are equal no matter what 
of the p number of memory cells might have its data 
30 inverted or fixed. As a result, the influences upon 
the neuron output values due to the fall of the data 
of one memory cell can be made lower than those of the 
general binary expressions. This expression method 
will be referred to as the "equivalent expression'* in 
35 the following. Here has been described the neuron 




output values, but the aforementioned equivalent ex- 
pression can naturally be used for the connection 
weigh ts • 

The binary expressions can naturally be used* 
This case is suited for expressing many values with 
less memory cells because the 2^ values can be expres- 
sed in the p bits. Fig. 10(a) shows one embodiment of 
the case, in which the binary expressions are used for 
the neuron output values and the connection weights. 
Only the memory cells on the data line of i = h (h = 
1, 2, - - and n) are shown in the array A, and only 
the memory cells on the data line of i = h and on the 
"word" line of s = f^^"(^^^^^ 1» 2, - and m-1) are 
shown in the array B. In Fig. lOCa), letters WT des- 
ignate a weighing circuit for weighing the signals of 
the memory cells and transmit them to the adders ah 
and bh. Here, the weighing coefficient are varied for 
the every memory cells, as shown, from 1 to 2^ for the 
neuron output values and from 1 to 2** for. the connec- 
tion weights. As a result, the neuron output values 
and the connection weights to be inputted to the mul- 
tiplier mh can take the 2^ and 2** numbers of values, 
respectively. The address selecting method for cal- 
culating the neuron output values may follow Fig. 8(d) 
like the embodiment of Fig. 9. Fig. 10(b) shows one 
embodiment of the characteristics of the nonlinear 
transfer function circuit D in the embodiment of Fig. 
10(a). The output gl al t.ernatel y repeats the values 0 
and 1, each time the input varies by (x2 - xl)/2^, 
and the output g2 alternately repeats the values 0 and 
1 for a period twice as long as that of the output gl . 
The periods are likewise varied by times so that the 
output gp is set from 0 to 1 across the value of (x2 - 
xl)/2. In other words, the nonlinear transfer func- 
tion circuit D may be so designed that it may operate 



9^ ..s. 



as an A/D converter* In the present embodiment, too, 
the nonlinear transfer function circuit D can be so 
designed that the neuron output values may increase 
nonlinearly for the input. In order to increase the 
neuron output values according to the sigmoid function 
in response to the input, for example, the periods for 
the individual outputs to vary may be decreased with 
the increase in the input, while the period ratios be- 
tween the different values g being held constant, so 
that the periods may be increased with the increase in 
the input when the input exceeds the value of (x2 - 
xl)/2. As has been described hereinbefore, according 
— to- the embodiments- shown in Figs; -10(a) and 10(b), the 
neuron output values and the connection weights can be 
expressed to have the 2^ values and the 2** values, 
respectively, by using the p and q numbers of memory 
cells, respectively. Thus, the embodiments are suited 
for giving the neuron output values and the connection 
weights multiple values with less memory cells. In- 
cidentally, in the present embodiment, too, various 
modifications can naturally be made such that the 
functions of the weight circuit WT and the adders al , 
- - and an, and bl , - - and bn are given to the 
multipliers by using multi-input circuits as the mul- 
tipliers ml, - - and mn . Here has been described 
the embodiments using the equivalent expressions and 
the binary notations. In addition, there are various 
methods such as the method of expressing a negative 
number with a code bit or a method of expressing data 
in plural bits, which can be separately used, if nec- 
essary . 

Next, here will be described an embodiment, in 
which a dynamic memory cell (i.e., DRAM cell) composed 
of one MOS transistor and one capacitor is used in the 
memory. 



Fig. 11 shows an embodiment in which the embodi- 
ment shown in Fig. 8(a) is composed of the DRAM cells. 
In Fig. 11, the array A and the array B are composed 
of: a plurality of intersecting data line pairs DAI 
and DA1\ " - and DAn and DAn , and DBl and SET, - - 

and DBn and DBn", and word lines WAl , WA2 , - - and 
WAm, and WBl , 1 . WB1,2, - - - WBl,n, and WB2 , 1 , WB2 , 2 , 
- - and WBm-l,n; and memory cells MC disposed at 
their intersections. Each memory cell MC is arranged 
at either of the intersections between the paired data 
lines and the word line- Thus, the embodiment has the 
so-called "folded-bit line structure". In Fig. 11, 
letters PR, SA, RSA and WS designate a precharge cir- 
cuit, a sense amplifier, a read sense amplifier, and a 
write switch, respectively, and correspond to the ar- 
ray control circuits 13A and 13B of Fig. 7(b). Let- 
ters MT appearing in the arithmetic circuit designate 
a multiplier. Numeral 16 designates a clock generator 
for generating clocks ^ and 4> b for controlling 
other circuits in response to addresses ADDA and ADDB 
and chip select signals CSA and CSB given from the 
outside of the chip. 

In the following, the operations of the embodi- 
ment of Fig. 11 will be described with reference to 
Fig. 12 and Figs. 13(a) and 13(b). Fig. 12 shows one 
embodiment between the operation modes and the exter- 
nal, signals. As has been described hereinbefore, the 
word lines of the memory cell arrays A and B are se- 
lected one by one in the earlier half of the arith- 
metic mode, and one word line of the memory cell array 
A is selected in the later half. In the memory mode, 
on the other hand, the memory cell arrays A and B are 
independently subjected to the reading and writing 
operations. In Fig. 12, the operation modes are fur- 
ther divided so» that those controls may be facilitat- 



ed^ Letters AR, AW, BR and BW in the memory mode des- 
ignate the reading mode from the array A, the writing 
mode in the array, the reading mode from the array B 
and the writing mode in the array B, respec t i ve I y • On 
the other hand, letters NR and NW in the arithmetic 
mode the earlier half for reading and calculating the 
individual data and the later half for writing the 
arithmetic results, respectively. In order to switch 
those six modes, the present embodiment uses four ex- 
ternal input signals: chip select signals CSA and CSB; 
write control signal WE; and ar i thme t i c ci rcui t con- 
trol signal NE. The chip select signals CSA and CSB 
-ass-rgn-the selecHons of the chips and the arrays A 
and B, The chips are unselected if both the signals 
CSA and CSB are at the H (i.e,, high) level; the array 
is selected if the signal CSA is at the L (i.e., low) 
level; and the array B is selected if the signal CSB 
is at the L (i.e., low) level. The write control sig- 
nal WE i s one for switching the write and read, as has 
been described hereinbefore, and establishes the read- 
ing operation at the H level and the writing operation 
at the L level. The signal NFalso establishes the 
memory mode at the H level and the arithmetic mode at 
the L level, as has been described hereinbefore. If, 
therefore, both the signals CSA and CSB are at the L 
level and the signal WE is at the H level whereas the 
signal NE is at the L level, for example, there is es- 
tablished the earlier'half mode NR of the arithmetic 
mode, in which both the arrays A and B are read out. 
Since the switching of the arrays A and B is assigned 
by the chip select signal, the address signal can 
divide the addresses into the address group ADDA for 
selecting the memory cells of the array A and the 
address group ADDB for selecting the memory cells of 
the array B. Here, the address group ADDA is the 
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generic name of the X-addresses for selecting the word 
lines of the array A and the Y-addresses for selecting 
the data lines of the array A. Likewise, the address 
group ADDB is the generic name of the X-addresses for 
selecting the word lines of the array B and the Y- 
addresses for selecting the data lines of the array B. 
In each operation mode, these address groups are ap- 
plied to the address pins in accordance with Fig* 12. 
According to the embodiment of Fig. 12 thus far de- 
scribed, the two chip select signals are provided to 
switch the arrays A and B thereby to separate the ad- 
dresses between the arrays A and B. Since the arrays 
A and B can be independently selected, it is possible 
to control each operation mode easily for selecting 
the arrays A and/ B. Incidentally, the relations be- 
tween the operation modes and the external signals can 
naturally be modified in various manners in addition 
to those of Fig. 12. For example, there can be adopt- 
ed: the method of adding addresses for switching the 
arrays A and B by using the chip select signal CS 
only; or the method of generating the X-address for 
selecting the word line of the array B in the mode NR 
from the counter disposed in the chip, by not dividing 
the addresses for the arrays A and B but selecting 
either the array A or B only. 

Fig. 13(a) shows an embodiment of the waveforms 
of Fig. 11 in the memory mode, and Fig. 13(b) shows an 
embodiment of the waveforms of Fig. 11 in the arith- 
metic mode. 

The operations of the memory mode are similar to 
the reading and writing operations of the ordinary 
DRAM. Fig. 13(a) shows the voltage waveforms in case 
the reading operations (in the mode AR) and the writ- 
ing operations (in the mode AW) are continuously ex- 
ecuted for the memory cell at the intersection between 



the word line WAl and the data line DAI in the array A 
in the memory mode. In Fig. 13(a), letters Vcc desig- 
nate a positive supply potential. Since, in the 
memory mode, the arithmetic circuit control signal NE 
is at the high level, the arithmetic circuit start 
signal 4> N is fixed at the low level so that the a- 
rithmetic circuit 12 is OFF. Before the start of the 
reading operation, signals FPA and PNA are set at 
Vcc/2 so that the sense amplifier SA is OFF, Since a 
precharge signal 4> r»>s. is at the high potential, on the 
other hand, the precharge circuit PR is turned on to 
short the data line pairs DAI and DAI , - - and DAn 
"and DAiT and to s~et""the potentiar at a precharge poten- 
tial VH. This precharge potential VH is set at Vcc/2 
according to the so-called "hal f -precharge method". 
When the chip select signal CSA drops to the low po- 
tential, the precharge signal * i»A. falls to turn off 
the precharge circuit PR so that the word line WAl 
selected by the address signal ADDA and a read Y- 
Select signal YRAl are transited to the high poten- 
tial. As a result, the MOS transistors of all the 
memory cells MC connected with the word line WAl are 
rendered conductive to establish individually delicate 
potential differences on the data line pairs DAI and 
DAI , - - -, and DAn and DAn in accordance with the 
electric charges stored in the capacitor. This poten- 
tial difference is read out and is detected by the 
sense amplifier RSA f ed wi th the Y-Select signal YRAl 
so that it is converted into the impedance difference 
of read lines OA and OA.^ This impedance difference is 
converted by the input/output circuit Into a voltage 
difference, which is amplified so that the content of 
the memory cell, i.e., the potential corresponding to 
1 or 0 is outputted as the read data DO. The so- 
called "rewriting operation" is executed in parallel 




with the aforementioned operations in the following 
manner. After the individual delicate potential dif- 
ference have been established in the data line pairs 
DAI and DAI , - - and DAn and DAn , the sijsjnal PPA is 
transited to the high potential whereas the signal PNA 
is transited to the low potential to start the sense 
amplifier SA. As a result, the delicate potential 
difference established in the data line pairs is am- 
plified to transit the data lines at the high poten- 
tial to the level Vcc and the data lines at the low 
potential to 0 V. As a result, the capacitors of all 
the memory cells MC connected with the word line WAl 
"are^wri t'teh'again wi W potential corresponding to 

the data before read. When the chip select signal CSA 
takes the high potential after the end of the rewrit- 
ing operation, the selected word line WAl and the read 
Y-select signal YRAl are transited to the low poten- 
tial, and the signals PPA and PNA are then transited 
to Vcc/2 to turn off the sense amplifier SA and tran- 
sit the precharge signal 4> to the high potential. 
As a result, the data line pairs are shorted, and the 
precharge potential VH is set with the potential, un- 
til the initial state is restored. The operations 
thus far described are the reading operations. 

Subsequently, the operations are shifted to the 
writing operations (in the mode AW) of the same cells. 
When, in the writing operations, the chip select sig- 
nal CSA takes the low potential and the write control 
input W takes the low potential, the data given to 
the write data DI are written in the selected memory 
cell in the array A. In the writing operation, too, 
the precharge signal ^ r»A is dropped at first to turn 
off the precharge circuit PR when the chip select sig- 
nal CSA drops to the low potential. Next, the word 
line WAl selected by the address signal ADDA and the 



Y-Select signal YRAl are transited to the high poten- 
tial. As a result, the MOS transistors of all the 
memory cells MC connected with the word line WAl are 
rendered conductive so that the delicate potential 
5 differences are individually established in the data 
line pairs DAI and DAI , - - and DAn and DAn in ac- 
cordance with the electric charges stored in the capa- 
citor. The delicate potential differences established 
in the data line pairs are amplified by the sense 
10 amplifier SA. Subsequently, an input circuit starting 
signal ^ wRi\ generated as a result of transition of 
the control input "WE to the low potential is transited 
^ to the high potential- As a result, the data given to 

J the write data DI are transmitted as the difference 

yl5 signals to write line pairs lA and Ta". Moreover, the 
n write Y-select signal YWAl is transited to the high 

n potential to turn on the write switch WS connected 

with the write destination memory cell. As a result, 
^ the write line pairs lA and TK are conducted to the 

^20 data line pairs DAI and DAI , respectively. As a re- 
□ suit, the data line pairs DAI and DAI are set to the 

^ potential corresponding to the data fed to the write 

data DI. After this, the input circuit starting sig- 
nal <f> wRiK is transited to the low potential, but the 
25 potential of the data line pairs is held by the sense 
amplifier SA. In the data line pairs for which the 
write switch WS is not turned on, the signal read out 
at first is amplified as it is by the sense amplifier 
so that the rewrite is executed. When the chip select 
30 signal CSA takes the high potential after the end of 
the rewriting operation, the selected word line WAl 
and the write Y-selection signal YWAl are shifted to 
the low potential. After this, the signals PPA and 
PNA are transited to Vcc/2 to turn off the sense 
35 amplifier SA and to transit the precharge signal <f> 
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to the high potential. As a result, the data line 
pairs are shorted and has its potential set to the 
precharge potential YH so that the initial state is 
restored. The operations thus far described are the 
5 writing operations. 

Here, the foregoing description is directed to 
the case in which a memory cell in the array A is con- 
tinuously subjected to the reading operation and the 
writing operation. Despite of this description, how- 

10 ever, one of the reading and writing operations can be 
continuously executed. Moreover, it is quite natural 
that memory cells in desired positions in a memory 
cell array, which are different for every reading ope- 
ration or writing operation, can be subjected to the 

15 reading operation or the writing operation by switch- 
ing the modes AR, AW, BR and BW. 

Next, the operations of the arithmetic mode will 
be described in the following. Fig. 13(b) shows the 
operation waveforms for obtaining a neuron output 

20 value Via* Let it be assumed that the necessary con- 
nection weights and neuron output values or the like 
have already been written by the writing operations in 
the memory mode. In order to establish the mode NR, 
first of all, the chip select signals CSA and CSB are 

25 set to the low level, and the write control signal WE 
is set to the high level whereas the arithmetic cir- 
cuit control signal NE is set to the low level. The 
addresses ADDA and ADDB are so set as to select the 
word line WAl of the array A and the word line WBl of 

30 the array B, Since the signals CSA and CSB are at the 
low level, the precharge signals 4> i.^ and * are 
transited to the low level. Since the signal NE is at 
the low level, the arithmetic circuit starting signal 
<f> N is transited to the high level. Subsequently, the 

35 word lines WAl and WBl are selected so that the neuron 



output values Vn, V21, - - and V„i and the connec- 
tion weights T^-ii, T^xa, - - and T'-m are read out 
of the memory cells on the word line WAl onto the data 
lines. Thus, the neuron output values read out from 
the array A and the connection weights read out from 
the array B are inputted to the multiplier MT which 
has been started by the arithmetic circuit starting 
signal * N, as shown in Fig- 11, In the multiplier 
MT, the data lines at the side of the array A and the 
data lines at the side of the array B are individually 
connected with the gates of the MOS transistors (Ti, 
Ta), which in turn are connected with the product/sum 
output line NO and the dummy line DM through the swi- 
tching MOS transistor (T3) fed with the arithmetic 
circuit starting signal * n. The product/sum output 
line NO has its one terminal connected with the power 
source VM through the load RMl , and the dummy line DM 
has its one terminal earthed to the ground. When the 
signal read out to the data lines is amplified to Vcc 
or 0 V by the sense amplifier SA, the electric current 
lows through the load RMl from the power source VM to 
the earthed electrode in the multiplier in which the 
product to the neuron output value and the connection 
weight is 1. As a result, the potential of the pro- 
duct/sum output line NO drops in proportion to the 
number of combinations, in which the product of the 
neuron output value and the connection weight is 1. 
The product/sum NO is inputted to the nonlinear trans- 
fer function circuit D- In this nonlinear transfer 
function circuit D, the summation of the products of 
the neuron output values and the connection weights is 
so high that the detection result of whether or not 
the potential of the product/sum output line NO is 
lower than reference voltage VR is outputted to a line 
NV, In the waveforms of the product/sum output line 



NO shown in Fig- 13(b), solid lines show the case, in 
which the result of the product sum is small, and 
broken curves show the case in which the result of the 
product sum is large. The input/output circuit de- 
5 tects the result of the nonlinear transfer function 
circuit D and outputs the neuron output value Via, 
which is to be subsequently written in the memory 
cells, to the write line pairs lA and Ta, Fig. 13(b) 
show the waveforms of the write line pair lA. This 
10 pair lA takes the high level, as indicated by a broken 
curve, in case the product sum is large, and the low 
level, as indicated by a solid curve, in case the pro- 
3 duct sum is small. The word line pair lA takes an op- 

1 posite phase. At the time when the neuron output 

ulS values are outputted to the write line pairs lA and 
H lA, a latch signal * l is transited to the high poten- 

ri tial- As a result, the potentials outputted to the 

^ write line pairs lA and Ta are latched by a latch cir- 

^ cuit which is disposed in the input/output circuit 10. 

U20 The latch signal *u may be raised with a delay till 
^ the signals appear in the pairs lA and lA, in response 

□ to the fall of the arithmetic circuit starting signal 

NE. Subsequently, the arithmetic circuit starting 
signal 4> N is transited to the low potential to turn 
25 off the arithmetic circuit so that the data lines are 
precharged as in the memory mode after the word lines 
have fallen. At this time, the latch signal is 
left at the high potential so that the neuron output 
values outputted to the write line pairs lA and lA are 
30 held. 

Next, the mode shifts to the mode NW, i.e., the 
later half of the arithmetic mode. First of all, the 
chip select signal CSA and the write control signal WE 
are set to the low level, and the chip select signal 
35 CSB is set to the high level to switch the address 



ADDA so that the memory cell for writing the neuron 
output value in the array A may be selected. The 
arithmetic circuit starting signal NE is left at the 
low level. As a result of the fall of the signal CSA , 
5 the precharge signal ^ pa drops to the low level to 

establish a state for the array A to be written. Sub- 
sequently, the potentials of the selected word line 
WA2 and the write Y-selection signal YWAl fall. As a 
result, the neuron output value V12 outputted to the 

10 write line pairs lA and TX is written in the memory 

cell which is connected with the WA2 and the data line 
DAI. Finally, all the potentials of the word lines 
are dropped for the' precharge • 'Since, moreover, the 
arithmetic circuit control signal NE falls, the latch 

15 signal 4> u falls to release the latch. Thus, prepara- 
tions are made for the next operations. The opera- 
tions thus far described are those in the arithmetic 
mode. All the neuron output val ues can - b|f calculated 
by continuing similar operations in different addres- 

20 ses in accordance with Fig. 8(d). 

In the structure thus far described, the circuit 
of the multiplier MT , which is connected with the 
dummy line DM, may be omitted. If, however, the gate 
capacitors or the like of the MOS transistors of the 

25 multiplier MT are added to the data lines only at one 
side, the data line capacities are unbalanced to trou- 
ble the operations of the sense amplifier, as the case 
may be. In this case', the structure of Fig. 11 could 
avoid the inferior influences coming from the unbal- 

30 ance of the data line capacities. 

Next, an embodiment of the circuit suitable for 
use in Fig. 11 is shown. Fig. 14(a) shows one embodi- 
ment of the nonlinear transfer function circuit D. 
The present embodiment is constructed of: a differen- 

35 tial amplifier composed of bipolar transistors Q720 



and Q719, a resistor R72 , and a MOS transistor Q721; 
and an inverter composed of an inverter INV75, MOS 
transistors Q715, Q716, Q717 and Q718, a resistor R71 
and a diode D71. The present circuit is started when 
5 the signal 4> ^ takes the high potential. Fig- 14(b) 

shows a relation between the potential of the product/ 
sum output line NO or the input of the nonlinear 
transfer function circuit D and the potential of the 
output NV. The output NV takes the high potential, if 
10 the potential of the product/sum output line NO is 

lower than the reference potential VR, but takes the 
low potential if the potential of the line NO is 
n higher than the reference potential VR, Since, ac- 

J^j cording to the present embodiment, the bipolar tran- 

L_yl5 sistors are used in the differential amplifier, it is 
H possible to realize a nonlinear circuit which is 

iri characterized to have a steep rise for the change in 

W the input. By setting the reference potential VR to a 

desired value, moreover, the characteristics of the 
fy20 nonlinear transfer function circuit D can be easily 
changed. Incidentally, the output of the different 
£5 amplifier cannot be made so high so as to avoid the 

saturation of the bipolar transistor Q719- As a re- 
sult, the downstream inverter may not operate if it is 
25 connected directly with the output of the differential 
amplifier. Therefore, the resistor R71 and the diode 
D71 are provided to drop the potential to be inputted 
to the MOS transistor' Q717, 

Fig, 14(c) shows one embodiment of the input/out- 
30 put circuit 10. The write circuit WR is composed, as 
shown in Fig, 14(c)., of an input buffer INBUF, write 
switches SWA and SWB, a latch circuit LAT , and in- 
verters INVIA and INVIB, The write switches SWA and 
SWB are used to switch which of the arrays A and B has 
35 its memory cells written with the write data DI . When 



the switching signal 4> wra is at the high potential, 
the write data DI is written through the input buffer 
INBUF in the memory cells of the array A by the write 
line pairs lA and lA. When the switching signal 4> wrs 
5 is at the high potential, the write data DI is written 
through the input buffer INBUF in the memory cells of 
the array B by the write line pairs IB and IB. The 
latch circuit LAT latches the data, which are out- 
putted to the output NV of the nonlinear transfer 

10 function circuit D in the arithmetic mode, to write 
them in the memory cells of the array A by the write 
line pairs lA and Ta. Since the potential relations 
of the output NV of the non 1 i near" transfer function 
circuit D and the write line pairs lA and lA are in 

15 phase, as is apparent from 14(c), the potential rela- 
tions between the product/sum output line NO of the 
nonlinear transfer function circuit D and the input 
common line lA are opposite to each other, as shown in 
Fig- 14(d), Since, in the embodiment of Fig. 11, the 

20 potential of the product/sum output line NO is the 

lower for the larger product sum of the neuron output 
values and the connection weights, as has been de- 
scribed hereinbefore, the circuit is made such that 
the potential relations between the product/sum output 

25 line NO and the input common line lA are opposite to 
each other. In case the potential of the product/sum 
output line NO is so designed that it may rise the 
more for the larger product sum of the neuron output 
values and the connection weights^ it is quite natural 

30 that the circuit may be so made that the potential 

relations between the product/sum output line NO and 
the input common line lA are in phase. 

Fig, 14(e) shows one embodiment of the read cir- 
cuit OUT. This read circuit OUT is composed of a 

35 current/voltage converter IVOUTl. a level shift cir- 



cuit LS, a read latch circuit OUTLT and an output buf- 
fer BUFOUT. In the curren t/ vo I t age converter IVOUTl, 
the data read out as the impedance differences to the 
read lines OA and OA and the read lines OB and OB are 
5 converted to the differences of the voltages of the 
lines OA* and OA* or read lines OB' and OB*. In the 
level shift LS , the voltage of the data read out from 
the current/voltage converter IVOUTl is shifted to a 
level, at which the bipolar transistors in the down- 

10 stream read latch circuit OUTLT are not saturated, to 
transmit it to the read latch circuit OUTLT- 

A detailed embodiment of the read latch circuit 
OUTLT is shown in Fig- 14(f). The read differential 
amplifiers AMPA and AMPB in the read latch circuit 

15 OUTLT are used to switch which of the data read out 

from the memory cells of the array A through the read 
lines OA and OA to lines LI and L2 and the data read 
out from the memory cells of the array B through the 
read lines OB and OB to the lines L3 and L4 are to be 

20 read out as the read data DO. When a switch signal ^ 
A is at the high potential, the data read out from the 
memory cells of the array A are outputted as the read 
data DO. When a switch signal 4> b is at the high 
level, the data read out from the memory cells of the 

25 array B are outputted as the read data DO. In the 

read latch circuit OUTLT, the bipolar transistor QIA 
is turned off, but the bipolar transistor QIB is 
turned on when the read latch circuit * is tran- 
sited to a higher potential than the voltage Vbs* As 

30 a result, the differential amplifiers AMPA and AMPB 

are turned off, and the differential amplifier AMPC is 
turned on. As a result, the read data are latched by 
the differential amplifier AMPC and the level shift 
circuit LSC. Specifically, according to the present 

35 embodiment, the read data DO can be latched and con- 
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tinuously outputted for a desired time period by tran- 
siting the read latch circuit * to a higher poten- 
tial than the voltage Vaa after the read data have 
been fixed. 

5 Incidentally, in the embodiment of the multiplier 

MT shown in Fig. 11, the data line pairs of the array 
A are connected with the gates of the MOS transistors 
which are farther from the earthed electrode than the 
data line pairs of the array B. As a result, the neu- 

10 ron output values and the connection weights are not 
equivalently handled when a product is to be taken. 
If this raises a problem, an embodiment of Fig, 15 may 
be used. In Fig. 15, the data line DAi is connected 
with the gates of MOS transistors Q7C3 and Q7C6 , and 

15 the data line pair DBi is connected with the gates of 
MOS transistors Q7C5 and Q7C4. Since the two data 
lines are connected with the MOS transistors closer to 
the earthed electrode and the MOS transistors farther 
from the same, the neuron output values and the con- 

20 nection weights are equivalently handled. As has been 
described hereinbefore, according to the embodiment 
shown in Fig. 11, the embodiment shown in Fig. 8(a) 
can be realized by using the DRAM cell which is com- 
posed of one MOS transistor and one capacitor. The 

25 DRAM cell can have its occupied area drastically 

reduced to attain a merit that it can be realized in 
high integration over the chip. In the DRAM cell com- 
posed of one transistor and one capacitor, although 
not touched in the foregoing description, refreshing 

30 operations are necessary within a constant time period 
for compensating the reduction of the stored charges 
due to the current leakage of the capacitor. In the 
present invention, too, the refreshing operations can 
be easily accomplished, if necessary, like the ordi- 

35 nary DRAM no matter which the mode might be the memory 
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mode or the arithmetic mode- 

In the aforementioned embodiment, the memory 
cells are exemplified by the DRAM cells but should not 
be limited thereto, but a similar data processing sys- 
5 tem can also be realized by using even other memory 
cells. Next, an embodiment using SRAM cells will be 
described in the following. Figs. 16(a) and 16(b) are 
circuit diagrams showing SRAM cells MCS, In the em- 
bodiment of the present invention, the SRAM cells MCS 

10 are used as the MC of Figs. 7(a) and 7(b), Fig. 8(a), 
Fig. 9(a) and Fig. 10(a). Either of the embodiments 
of Figs. 16(a) and 16(b) is advantageous In that the 
controls are far easier than the case of using the 
DRAM cells because it requires neither rewriting nor 

15 refreshing operation unlike the DRAM cells. Fig. 

16(c) shows one embodiment for realizing the embodi- 
ment of Fig. 8(a) by using the SRAM cell of Fig. 16(a) 
or 16(b). In Fig. 16(c), letters MCS designate the 
SRAM cell, and letters LD designate a data line load. 

20 Figs. 17(a) and 17(b) show examples of the operation 
waveforms. Fig. 17(a) shows an example of the case, 
in which cells connected with the data lines DAI and 
DaI and the word line WAl are continuously subjected 
to the reading operations and the writing operations 

25 in the memory mode. Fig. 17(b) shows an example of 
the operation waveforms of the case, in which the 
neuron output valve Via is to be calculated in the 
arithmetic mode from both the neuron output values 
Vii, Vai, - - -, and Vnx stored in the memory cells of 

30 the word line WAl and the connection weights T^n, 
T^ai, - - and T^xn stored in the memory cells of 
the word line WBl . The basic operations are similar 
to those of the aforementioned case of the DRAM cells, 
and their description will be omitted. The SRAM cell 

35 has a merit that its control is simpler than the DRAM 



cell, because it does not need the rewriting operation 
and the refreshing operation- Because of no necessity 
for the rewriting operation, moreover, there is 
another merit that the reading and writing speeds in 
5 the memory mode and the cycles in the arithmetic mode 
can be accelerated- 

The description thus far made is directed to the 
example of the circuit structure, in which the embodi- 
ment of Fig. 8 is realized by us i ng . the DRAM cells and 

10 the SRAM cells. Next, here will be described an ex- 
ample of the circuit structure for expressing the neu- 
ron output values and the connection weights by using 
a plurality of memory cells* Although the embodiment 
to be described uses the DRAM cells, the present in- 

15 vention can be likewise realized even by using teh 
SRAM cells. 

Next, here will be described an example of the 
circuit structure in which the neuron output values 
are expressed by using the DRAM cells and in which the 

20 connection weights are expressed by using a plurality 
of memory cells. In Fig, 18(a), the data line pairs 
DAll and DaTT, DA12 and DA12 , - - and DAIP and DAIP 
in the array A correspond to the data line pairs in 
the array A, which are to be inputted to the adder al 

25 in Fig. 9(a). On the other hand, the data line pairs 
DAnl and DAnl , and DAn2 and DAn2, - - and DAnP and 
DAnP in the array A correspond to the data line pairs 
in the array A, which' are to be inputted to the adder 
an in Fig. 9(a). The array B has similar correspon- 

^0 dences. As shown in the input/output circuit DIOlO, 

there are provided an r number of input terminals DOl , 
- - -, and DOr and an r number of output terminals 
DIl, - - and DIr (wherein r is the larger number of 
P and q) so that the data of p bits or q bits indicat- 

^5 ing the neuron output values or the connection weights 



may be simultaneously read out or written in the memo- 
ry mode. In the array A in the arithmetic mode, the 
data of every p bits read out to the data lines by 
selecting the word lines are synthesized by the adder 
5 ADD to output the neuron output values to the neuron 
output value output lines VOl , V02 , - - and VOn • 
In the array B, on the other hand, the data of q bits 
read out to the data lines by selecting the word lines 
are synthesized by the adder ADD to output the connec- 
10 tion weights to connection weight output lines TOl , 

T02 , - - and TOn. These values are inputted to the 
BLK2 so that the resultant product sum is inputted to 
the nonlinear transfer function circuit DIO, The out- 
.5 put of the nonlinear transfer function circuit DIO 

Hs corresponding to the neuron output value is trans- 

mitted to the input/output circuit DIOlO and latched 
by the latch signal 4> l. Subsequently, the address is 
^ switches to select the p number of cells to write the 

H determined neuron output value, and the write Y-select 

!^0 signal YWAi is raised to write the latched neuron out- 
ry put values in parallel in the p number of selected 

cells. By continuing these operations, the neuron 
R output values can be updated like the embodiment of 

Fig. 11(a). According to the present embodiment, the 
25 embodiment of Fig. 9Ca) can be realized by equivalent- 
ly adding the data of the plural memory cells inputted 
to the adder ADD- By weighing and adding bit by bit 
the data of the plural memory cells inputted to the 
adder ADD, moreover, it is possible to realize the em- 
30 bodiment of Fig. 10(a), in which the neuron output 

values and the connection weights are expressed with 
binary numbers of plural bits. Since the present em- 
bodiment can also be applied to the case in which the 
neuron output values and the connection weights are 
35 expressed with a plurality of bits by another method, 




a variety of data processings can be accomplished in 
accordance with the purpose* Since the DRAM cells are 
used in the present embodiment, a hi^h integration can 
be achieved. Since, moreover, the data of the plural 
5 memory cells are processed in parallel both in the 

memory mode and in the arithmetic mode, the data pro- 
cessing can be executed at a high speed like the case 
of expressions with 1 bit, although the neuron output 
values and the connection weights are expressed by the 

10 plural bits. Here, In the BLKl , the signals of plural 
memory cells are synthesized by the adder, and the 
result is inputted to the BLK2 acting as the product/ 
sum circuit. However, a variety of modifications can 
be made by omitting the addition at the BLKl such that 

15 the data of the plural memory cells indicating the 

neuron output values or the connection weights are in- 
putted in parallel to the product/sum circuit BLK2 so 
that they may be subjected to the multiplications and 
summat i ons , 

20 In the following. Fig. 18(b) for realizing the 

embodiment of Fig. 9, in which the neuron output 
values and the connect i on wei gh ts are expressed in a 
plurality of equivalent bits by the embodiment shown 
in Fig. 18(a), shows one embodiment of the BLKl of 

25 Fig, 18(a). Here is shown the BLKl which is connected 
with the data lines DAll, - - and DAIP of the array 
A. The same circuit can a.lso be used in another BLKl 
of the array A. The circuit of the present embodiment 
can also be used in the array B, if the number of the 

30 data line pairs, the read line pairs or the write line 
pairs is changed from p to q and if a q number of cir- 
cuits each having a p number of precharge circuits PR 
are provided. In the present embodiment, there are 
provided p pairs of read line pairs OAl and OAl , - - 

35 and OAp and OAp and p pairs of write line pairs lAl 
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and lAl, - - and lAp and lAp so that the p number 
of memory cells may be subjected to the writing or 
reading operations in parallel. The read sense ampli- 
fier RSA and the write switch WS are consecutively 
5 connected in the same BLKl , as shown, with the read 

line pairs OAl and OAl , - - and OAp and OAp and the 
p pairs of write line pairs lAl and lAl, - - and 
lAp and lAp, For one pair of read or write lines, 
specifically, every p pairs are connected with the 
10 data line pair. The adder ADD is composed of a load 
circuit LD103 and a p number of voltage/current con- 
verters VI. In the voltage/current converter VI, the 
3 data lines DAll, DA12, - - and DAlp are connected 

Q with the gates of the MOS transistors, which in turn 

are connected in series with the MOS transistors hav- 
5 ing their gates fed with the arithmetic circuit start- 

2 ing signal 4> n. to connect the earthed electrode and 
o the neuron output value output line VOl • This neuron 

output value output line VOl is connected through a 
y20 resistor in the load circuit with the power source 
y VMOl . As a result, if the amplification of the data 

5 line potential is ended in the state started by the 

3 arithmetic circuit starting signal 4> n, the potential 
of the neuron output value output line VOl is dropped 

25 by a voltage proportional to the number of data lines 
which are raised to the high potential, i.e., Vcc. 
According to the present embodiment, therefore, the 
neuron output values can be expressed in terms of the 
potential drop of the neuron output value output line 

20 VOl. Incidentally, the provision of similar circuits 
at one side of the data lines DAll, - - and DAlp is 
to avoid the unbalance of the data line capacities for 
the same reasoning as that of the multiplier MT of 
Fig. 11(a). According to the embodiment thus far de- 

35 scribed, the neuron output values or the connection 



weights expressed by the plural memory cells can be 
read out to the neuron output value output line or the 
connection wei^:ht output line. 

Fig. 18(c) shows one embodiment of the block BLK2 
5 for calculating the product sum of the neuron output 
values and the connectio'n weii^hts and the nonlinear 
transfer function circuit DIO, In Fig. 18(c), the 
block BLK2 is composed of the load circuit LD102 and 
the multiplier MTIO. The neuron output value output 

10 lines VOl , V02 , - - and VOn and the connection 

weight output lines TOl , T02 , - - and TOn are con- 
nected with the gates of the MOS transistors M16cl and 
-M16c2 of the MTIO, and the MOS transistors are con- 
nected in parallel with the MOS transistor M16c3, 

15 which has its gate fed with the arithmetic circuit 

starting signal 4> Nt to connect the earthed electrode 
and the product/sum output line NO. On the other 
hand, the product/sum output line NO is connected 
through the resistor R02 in the load circuit LD102 

20 with the power source VM02 . In the state in which the 
arithmetic circuit starting signal * n is at the high 
level so that the present circuit is started, the 
potential of the product/sum output line NO is dropped 
the more for the larger sum of the products of the 

25 potentials of the corresponding neuron output value 

output lines VOl , V02 , - - and VOn and the connec- 
tion weight output lines TOl, T02 , - - -, and TOn- As 
has been described hereinbefore, the potentials of the 
neuron output value output lines VOl , V02 , - - -, and 

30 VOn and the connection weight output lines TOl, T02 

and TOn are dropped substantially in proportion to the 
magnitudes of the neuron output values and the connec- 
tion weights so that the potential of the product/sum 
output line NO becomes the higher for the larger pro- 

35 duct sum of the neuron output values and the connec- 
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tion weights. The product/sum output line NO is in- 
putted to the nonl inear transfer function circuit DIO. 
The nonlinear transfer function circuit DIO can be 
constructed by connecting an n number of circuits 
5 shown in Fig, 18(d) in parallel. The circuit of Fig. 
18(d) is made like the nonlinear transfer function 
circuit D of Fig. 14(a) by combining the differential 
amplifier and the inverter. Since, however, the 
polarities of the product/sum output line NO and the 

10 product sum of the neuron output values and the con- 
nection weights are different between the embodiments 
of Fig. 11 and Figs. 18(a), 18(b) and 18(c), the re- 
sistor Rx of the differential amplifier of Fig, 18(d) 
is connected in the position opposed to that of the 

15 resistor R72 of Fig. 14(a). In Fig. 18(d), therefore, 
the output NVx transits to the high potential if the 
product/sum output line NO exceeds the reference volt- 
age VRx (X = 1, 2, - - and p). If a p number of 
such nonlinear transfer function circuits DSx are 

20 provided and if the reference voltage VRx is changed, 
as shown in Fig. 18(e), the change in the product/sum 
output line NO can be indicated by the number of such 
ones of the p outputs NVx as take the high potential. 
According to the present invention, the characteris- 

25 tics of the nonlinear transfer function circuit can be 
easily varied by varying the value of the reference 
voltage VRx. Incidentally,, in case the circuit shown 
in Fig. 18(c) is used' as the multiplier MTIO, the 
potential variation of the product/sum output line NO 

30 is generally kept away from linearity for the mag- 
nitudes of the product sum of the neuron output values 
and the connection weights by the characteristics of 
the MOS transistors. It is, therefore, advisable to 
set the value of the reference voltage VRx by con- 

35 sidering the characteristics of the multiplier or the 



adder so that the characteristics of the nonlinear 
transfer function circuit may take a desired shape. 
As the case may be, the characteristics of the in- 
dividual chips may be made difficult to understand ac- 
5 curately because of the fluctuations of the production 
conditions. In this case, the known neuron output 
values and connection weights are actually written in 
the arrays A and B, and the potential of the product/ 
sum output line NO in the arithmetic mode is measured 

10 so that the vale of the reference voltage VRx may be 
resultantly trimmed to the desired characteristics. 

Incidentally, here will be omitted the detail of 
the input/output circuit DIOlO of Fig. 18(a). The 
circuit of reading or writing a plurality of memory 

15 cells in parallel can be easily realized by using a 
plurality of circuits which are similar to the read 
circuit OUT or the write circuit WR shown in Figs. 
14(c), 14(e) and 14(f). Moreover, the structure of 
the clock generator 16 will be omitted but can be 

20 easily realized like, the circuit used in the ordinary 
memory . 

Next, the method for realizing the embodiment of 
Fig. 10, in which the neuron output values and the 
connection weights are binarily expressed in plural 

25 bits, will be described in connection with the embodi- 
ment of Fig. 18(a) • In order to add the data expres- 
sed binarily with plural bits, as shown in Fig* 10(a), 
it is necessary to we'ight and add the data of the 
plural memory cells bit by bit. For this necessity, 

30 the potential of the neuron output value output line 

vol drops in proportion to the magnitude of the binary 
neuron output values if the ratios of the gate width 
of the MOS transistors connected with the data lines 
in the voltage/current converters VII, VI2, - - and 

35 vip in Fig. ia(b) are 1:2:4:,---, and : 2^. 



If, therefore, similar circuits are used for other 
neuron output values or connection weights, the wei;?h- 
ing additions can be realized, as shown in Fig* 10(a). 
The block BLK2 shown in Fig* 18(c) can be used as it 
5 is as the multiplier. The nonlinear transfer function 
circuit has to be given the function of the AD con- 
verter for rewriting the arithmetic result outputted 
to the product/sum output line NO in the plural memory 
cells in the binary notations. For this necessity, it 
10 is possible to use the embodiment shown in Fig. 19(a), 
In the embodiment of Fig. 19(a), a z (z = 2^) number 
of nonlinear transfer function circuits DSl, DS2 , - - 

and DSz and an encoder are combined. These non- 
linear transfer function circuits DSl, DS2, - - and 
^"45 DiSz are given the characteristics shown in Fig- 19(b) 
^5 by adjusting the reference voltage VRx with the cir- 

^4 cuit of Fig. 18(d). Then, the magnitude. of the pro- 

^ duct sum of the neuron output values and the connec- 

5 tion weights can be known like the embodiment of Fig. 

1720 18(c) from the number of ones of the outputs NAl , NA2 , 
ry - - and NAz as have the high potential. Then, the 

equivalent expressions of z bits have to be changed 
f5 into binary expressions of p bits by the encoder so 

that they have to be transmitted to the write circuit 
25 through the p number of output lines NVl , NV2 , - - 
and NYp. It follows that the encoder. of Fig. 19(a) 
may be given the input/output relations shown in Fig. 
19(c). This encoder 'can be realized without dif- 
ficulty. An example of the structure for p = 3 is 
30 shown in Fig. 19(d). The present embodiment can be 
easily extended to the cases other than that for p = 
3. 

The description thus far made is exemplified by 
the mul t i - 1 ayered neural network. Despite of this ex- 
35 empl i f icat ion , however, the present invention should 



not be limited to the mul t i - 1 ayered neural network but 
can be applied to other types of networks by using the 
embodiments thus far described. Figs. 20(a) and 20(b) 
and Figs. 21(a) and 21(b) show embodiments for realiz- 
5 ing the data processing using the Hopfield network ac- 
cording to the algorithm of Fig. 5(b). Fig. 20(a) 
shows an embodiment, in which the unsynchroni zed Hop- 
field network is realized by using memory cells one by 
one for expressing the neuron output values and the 

10 connection weights. As has been described with refer- 
ence to Figs. 2 and 3, the basic arithmetic method is 
commonly shared between the mul t i - 1 ayered network and 
the Hopfield network. In the Hopfield network, how- 
ever, the arithmetics are carried out by using the 

15 neuron output values from all the neurons including 
those of itself. In Fig. 20(a). therefore, all the 
neuron output values are stored in one word line of 
the array A. In the array B, as shown, the connection 
weights necessary for calculating one neuron output 

20 value are stored on a common word line. The updating 
of the neuron output values can be executed in the 
following manner. In order to update the neuron out- 
put value Vi, for example, the word line WA of the ar- 
ray A and the word line of j = 1 of the array B are 

25 raised. As a result, the new neuron output value of 
gdiiVi + TiaVa + - - - + TinVn) is Calculated. This 
value may be written in the memory cell which is lo- 
cated in the position' of f = 1 on the word line WA of 
the array A. The updatings of the other neuron output 

30 values are similar. The value V^, for example, is up- 
dated by raising the word line WA of the array A and 
the word line of j = 4 of the array B. As a result, 
the new value of g(T4iVi + T^aV^ + - - - + T^nVn) 
is calculated. This value may be written in the 

35 memory cell in the position of i = 4 on the word line 



WA of the array A. Thus, the arithmetics of the un- 
synchron i zed Hopfield network can be executed by up- 
dating the neuron output values Vi in the desired or- 
der. The arithmetics of the synchronized Hopfield 
network can be easily realized by using the memory 
cells on the word line WAl of the array A for storing 
the neuron output values at present and by using the 
memory cells on the word line WA2 for storing the new 
neuron output values, as shown in Fig. 20(b). First 
of all, the word line WAl of the array A and the word 
line of j = 1 of the array B are raised. As a result, 
the new value Vi of g(TixVi + TiaVz + - - - + TxnV„) 
is calculated. This value may be written in the 
memory cell in the position of i = 1 on the word line 
WA2 of the array A. Subsequently, the neuron output 
values Va, Va, - - and Vn are updated and written 
in the memory cells on the word line WA2 of the array 
A. When the updatings of all the neuron output values 
are ended, the updating of the neuron output values is 
continued by interchanging the roles of the word lines 
WAl and WA2 of the array A such that the word line WA2 
is selected for calculating the neuron output values 
whereas the word line WAl is selected for storing the 
neuron output values. From now on, the processings 
are likewise proceeded by interchanging the roles of 
the word lines WAl and WA2 of the array A. Thus, ac- 
cording to the embodiment -of Fig. 20(b), the arith- 
metics of the . synchron i zed Hopfield network can be ex- 
ecuted. 

Likewise, the Hopfield network can be realized by 
using a plurality of memory cells for expressing the 
neuron output values and the connection weights. Fig, 
21(a) shows an embodiment for realizing the unsyn- 
chronized Hopfield network by using Ihe p and q num- 
bers of memory cells equivalently for expressing the 



neuron output values and the connection weights- Like 
Fig. 20(a), all the neuron output values are stored in 
one word line of the array A. Here, the p number of 
cells express one neuron output value. The storage of 
5 the array B is made such that the connection weights 
necessary for calculating one neuron output value are 
arrayed on a common word line for every q number of 
cells. The updating of the neuron output values may 
be executed like the embodiment of Fig* 20(a)* Since, 

10 however, the p number of memory cells are individually 
used for expressing the neuron output values, a p num- 
ber of output lines of the nonlinear transfer function 
circuit D are provided so that the arithmetic results 
may be written in parallel in the p number of cells. 

15 The synchronized Hopfield network can also be easily 
realized like Fig. 21(b) if two word lines of the ar- 
ray A are used like Fig* 20(b). Likewise, it is quite 
natural that the synchronized and unsynchron i zed Hop- 
field networks can be realized by binary expressions 

20 using p and q numbers of memory cells for expressing 
the neuron output values and the connection weights, 
as shown in Fig. 10(a). 

Figs. 8(a) and Figs. 20(a) and 20(b), and Fig. 
9(a) and Figs. 21(a) and 21(b) present basically iden- 

25 tical structures. If, therefore, the embodiments of 
Figs. 11 to 19, the data processings according to the 
embodiments of Figs. 20(a) and 20(b) and Figs. 21(a) 
and 21(b) can be easily realized. Incidentally, in 
the Hopfield network, the procedure of continuing the 

30 updating of the neuron output values falls in the so- 
called "local minimum", in which the energy is not the 
minimum but the minimal, so that the neuron output 
values are not varied any more. In order to avoid 
this, the well-known "quas i -anneal i ng method" can be 

35 used. The method of changing the shape of the non- 



linear transfer function gradually is known for 
realizing the quas i -anneal i ng method, as described on 
pp, 122 of Neural Network Processing (published by 
Sangyo Tosho and edited by Hideki Asou) * According to 
5 the present invention, this method can be easily real- 
ized by switching a plurality of nonlinear transfer 
function circuits D having different characteristics 
and by controlling the characteristics of the non- 
linear transfer function circuits D from the outside. 
10 Although there have been described examples in 

which the neuron output values and the connection 
weights have been handled as positive numbers mainly 
in the mul t i -layered or Hopfield network, it may be 
convenient depending upon the application that both or 
rilS one of the two values can take positive and negative 
In values- The present invention can be easily applied 

to such case. Fig. 22 shows one embodiment of the 
l£ present invention, in which both the neuron output 

= values and the connection weights are enabled to take 

fy20 positive and negative values. In Fig. 22, the neuron 
fU output values are stored in the memory cell array A, 

;t and the connection weights are stored In the memory 

E cell array B. The individual values are expressed 

with p or q bits indicating the absolute magnitudes 
25 and with 1 bit Indicating the codes. The bits in- 
dicating the codes (as will be called the "coding 
bits") indicate a positive value with "1" and a nega- 
tive value with "0". 'Of the neuron output values and 
connection weights thus read out by the method similar 
30 to those described herein before, the portions of the 
p or q bits indicating the absolute values are in- 
putted to the adders al , - - -, and an and bl, - - 
and bn so that the resultant analog values are in- 
putted to the multipliers ml, - - and mn . Inciden- 
35 tally, when the neuron output values and the connec- 



tion weights are to be expressed in the binary nota- 
tion, the data of the individual p and q bits inputted 
to the aforementioned adders al , - - and an, and bl 

- - and bn may be weighed and inputted like Fig, 

5 10. On the other hand, the coding bits are inputted, 
as shown in Fig. 22, to exclusive OR circuits EORl , - 

- and EORn. In case the coding bits fail to become 
similar, namely, when the result of multiplications is 
negative, the outputs of the aforementioned exclusive 

10 OR circuits take the high level. In case of the simi- 
larity, namely, when the multiplication result is 
positive, the outputs of the exclusive OR circuits 
take the low level* Switches SWl , - - and SWn 
operate to transfer the outputs of the multipliers to 

15 the adders cl , - - and cn, when the outputs of the 
exclusive OR circuits take the low level, and the same 
to the multipliers cl * , - - and cn * when the out- 
puts of the exclusive OR circuits take the high level. 
As a result, the sum of the positive results of multi- 

20 plications is outputted to the product/sum output line 
NO, and the sum of the negative multiplication results 
is outputted to the product/sum output line NO ' • In 
the nonlinear transfer function circuit D, the dif- 
ference between the signals of the product/sum output 

25 line NO and the product/sum output line NO' is con- 
verted into a digital value of p bits and fed to the 
bus ABS so that the coding bits are determined accord- 
ing to the magnitudes of the signals of the product/ 
sum output line NO and the product/sum output line NO' 

30 and outputted to a bus SIGN. Incidentally, it is 

easily possible according to the method similar to the 
aforementioned ones to give the nonlinear characteris- 
tics, as shown in Fig. 9 or 10, according to the ex- 
pressions of the neuron output values. According to 

35 the present embodiment, both the neuron output values 



and the connection wei^chts can take the positive and 
negative values. As a result, the present embodiment 
is advantageous in that the range for applying the 
data processing is extended. Although both the neuron 
5 output values and the connection weights are enabled 

to take the positive and negative values, it is easily 
possible to make modifications such that either of 
them take the positive values- 

The description thus far made has been directed 
10 to the embodiment in which the product/sum function 

necessary for calculating the neuron output values and 
the nonlinear transfer function circuit are realized 
as the arithmetic circuit. Despite of this descrip- 
^5 tion, however, a circuit for other arithmetics can be 

H5 added to the arithmetic circuit. For example, the 
^2 data processing system according to the present inven- 

^4 tion can be applied to the so-called "classification 

JJi problem" such as the speech recognition or the letter 

5 recognition, in which input patterns are classified 

t!?0 into several classes- In this case, the comparator is 
rO conveniently disposed in the arithmetic circuit, as 

?3 has been described hereinbefore- In the classifica- 

J5 tion problem, a desired value corresponding to a cer- 

tain class can be attained as the output in case the 
25 inputted patterns are apparently classified to the 

class- In the delicate case, however, in which it is 
questionable to determine which of plural classes the 
inputted pattern belongs to, the classification may 
fall at a middle between the desired values of the 
30 plural clas.ses- In case the inputted speech is 'K* in 
the speech recognition, for example, it is coded, and 
the connection weight is so set that the neuron output 
value (or the desired value) of 1111 may be obtained 
in the output layer for the speech waveforms given to 
35 the input layer- If the input is 'C, the connection 



weight is so set that the output value (or the desired 
value) of 0000 may be issued. If. in this case, a 
middle speech waveform between the *K* and the , 
the neuron output value of the output layer may output 
5 a middle value such as 0001 or 1110. In this case, 

the distance (or similarity) between the neuron output 
value of the output layer and the desired value of 
1111 for the 'K' or the desired value 0000 for the VC / 
can be interpreted as measures for giving the similar- 
10 i ty to the 'K' or 'C* of the input speech. It is, 

therefore convenient to give a function to determined 
the distance between the output result and the desired 
value by providing the arithmetic circuit with a cir- 
^ cuit for comparing the neuron output value of the out- 

put layer and the desired value of the class. 
Jn Fig. 23 shows one embodiment, in which there are 

"'^ integrated in one semiconductor chip: an arithmetic 

Jfi circuit 12a for comparing the neuron output values and 

- the desired value; and an . ar i thme t i c circuit 12b for 

17-20 calculating the neuron output values. In Fig. 23: 
lU the desired value is stored in the memory TG; the neu- 

:= ron output values are stored in the memory A; and the 

n connection weights are stored in the memory B. The 

calculations of the neuron output values may be ac- 
25 complished by the method similar to those thus far 
described, by reading out the neuron output values 
from the memory A and the connec t i on we i gh ts from the 
memory B, by calculating the neuron output values with 
the arithmetic circuit 12b and by writing the calcu- 
30 lated result in the memory A. The comparisons are 

carried out by reading the neuron output values from 
the memory A and the desired value from the memory TG, 
by determining the distances in parallel with the 
arithmetic circuit 12B, and by writing the result in 
35 the memory TG or outputting the same through the 



input/output device. Since, in the present embodi- 
ment, both the memories TG and A and the arithmetic 
circuit 12a are formed over the common chip, the num- 
bers of the buses 1 and 2 can be easily increased to 
process the numerous bits in parallel. This results 
in an advantage that the distances can be calculated 
at a high speed. Incidentally, in the structure thus 
far described, it is convenient to divide the arith- 
metic mode into the neuron output value- calculating 
mode for calculating the neuron output values and the 
comparison mode for comparing the neuron output values 
and the desired value to determine the distance. The 
switching of the arithmetic mode can be accomplished 
in response to the two arithmetic circuit control sig- 
nals NeT and NE2, for example. Specifically: th^e 

memory mode may be selected if both the signals NEl 
and NE2 are at the high level; the neuron output value 
calculating mode may be selected if the signal NEl is 
at the low level whereas the signal NE2 is at the high 
level; and the comparison mode may be selected if the 
signal "NET is at the high level whereas the signal NE2 
is at the low level. Incidentally, in the embodiment 
of Fig. 23, the memory is divided into three whereas 
the arithmetic circuit is divided into two, but these 
divided circuits may naturally be mixed over the chip. 
As has been described hereinbefore, according to the 
present embodiment, it is. possible to determine the 
distances between the neuron output values and the 
desired value at a high speed. As a result, the data 
processing speed can be accelerated in case it is 
necessary, as in the pattern recognition using the 
multi-layered network, to compare the neuron output 
values and each desired value to determine the dis- 
tances inbetween. 

Fig- 24 shows one embodiment of the arithmetic 



circuit 12a of Fig. 23, i.e., a circuit for comparing 
the neuron output values of the output layer and the 
desired value to calculate the humming distances in- 
between. In the following, it is assumed that the 
5 memories TG and A of Fig- 23 are of the type, in which 
the data of the memory cells are read out to the data 
line pairs, as in Fig, 11, Fig- 16 or Fig. 18, and 
that the memories have the arrays TG and A, respec- 
tively. The circuit of Fig. 24 is composed of a com- 

10 parator CMP and a comparison result converter COMPOUT. 
The comparator CMP is composed of a comparator CMPU 
and a load resistor Rcmp connected in parallel, and 
the comparison result converter COMPOUT is composed of 
differential amplifiers AMP211, AMP212, - - and 

15 AMP21Z. The comparator CMP is connected with the data 
lines DTGl and DTGl , - - and DTGr and DTGr of the 
array TG, and the data lines DAI and DAI,. - - -, and 
DAr and DAr of the array A. Here, letter r designates 
the number of memory cells on one word line and takes 

20 the value of n, if the neuron output values are ex- 
pressed in 1 bit, and the product of n and p if the 
neuron output values are expressed in p bits. Accord- 
ing to the present embodiment, it is possible to cal- 
culate the humming distances between the data read out 

25 onto the data lines DTGl and DTGl , - - - ,. and DTGr and 
DTGr of the array TG and the data read out onto the 
data lines DAI and DAI , - - and DAr and DAr of the 
array A. The operations of the present embodiment 
will be described in the following. First of all, a 

30 clear signal 4>o is raised in advance to turn on a MOS 
transistor Q216 and to break the gate voltage of a MOS 
transistor Q215. After the clear signal <J> o has been 
broken so that the signal is read out to the data line 
to set the data line potential to Vcc or 0 V, the com- 

35 parator is started by a comparator starting signal 



OMP. Then, the logic of exclusive OR is taken in each 
of the groups of the data lines (DTGl , DAI), (DTG2, 
DA2), - - and (DTGr. DAr) connected with the com- 
parator. As a result, the gate of the MOS transistor 
5 Q215 is left at the low potential, in case the data 

are similar on the data lines at- the array TG and the 
data lines at the array A, but otherwise is transited 
to the high potential- Then, the MOS transistor Q215 
is turned on in the comparator CMPU in which the data 
10 are not similar between the data lines of the array TG 
and the data lines of the array A. As a result, the 
more current will flow from the power source VCMP 
through the load resistor ROMP to the earthed elec- 
ta trode for the larger number of groups of the data 
J^IS lines (DTGl , DAI), (DTG2, DA2 ) , - - and (DTGr, DAr), 
J| in which the data are not similar. As a result, the 

potential of the compare line CO will be the lower for 
fQ the larger number of groups in which the data are not 

3 similar. The compare line CO is connected with the 

[n20 differential amplifiers AMP211, AMP212, - - and 
fU AMP21Z" disposed in the comparison result converter 

J2 COMOUT- If the reference voltages VRCl , VRC2 , - - 

n and VRCZ of those differential amplifiers are set to 

suitable values, the number of those of the comparison 
25 result output lines DCOl , DC02 , - - and DCOZ, which 
take the high potential, is the larger for the larger 
drop of the potential of the compare line CO. In 
other words, the comparison result converter COMPOUT 
operates as a kind of AD converter. Thus, according 
30 to the embodiment of Fig. 24, the data read out to the 
plural data lines of the array TG and the data read 
out to the plural data of the array A can be compared 
to determine their humming distance. If, therefore, 
one word is selected from each of the array TG and the 
35 array A, the data stored in the memory cells on the 



selected word lines can be compared with each other- 
If, therefore, the desired values are individually 
stored in the memory cells on the array TG, it can be 
compared with the neuron output values, which are 
stored in the memory cells on one word line of the ar- 
ray A, to know what desired value the neuron output 
values are close to and how close they are. In case, 
therefore, the obtained neuron output values are not 
similar to the desired value corresponding to the 
class, it is possible to know at a high ^peed what 
class the neuron output values are close to and how 
close they are. 

IncidentaHyr-in- -the embodiment of Fig. 24, the 

result outputted to the comparison result output lines 
may be outputted to the outside of the chip through 
the input/output circuit at each time of comparison. 
Alternatively, the capacity of the memory TG may be 
made larger than that necessary for storing the desir- 
ed values so that the results may be once written in 
the memory TG and then outputted altogether. 

Finally, an embodiment for further speeding up 
the system of the present invention by using a regis- 
ter will be described in the following. As has been 
described hereinbefore, according to the present in- 
vention, the neuron output values are calculated by 
reading out the necessary data from the memory, by 
determining the neuron output values with the arith- 
metic circuit, and by' rewriting the determined result 
in the memory. In other words, one arithmetic mode 
(i.e., the neuron output value arithmetic mode) cycle 
is composed of the reading operation and the writing 
operation, and the arithmetic circuit is inactive in 
the writing operation. If, therefore, the time pe- 
riod, for which the arithmetic circuit is in active, 
is shortened, the arithmetic mode can be further 
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speeded up. Fig. 25 shows one embodiment in which the 
arithmetic mode is speeded up on the basis of the 
aforementioned point of view. The embodiment of Fig. 
25 is made by adding the register and the switches 
5 SWl , - - and SWr to the embodiment of Fig- 7. Ac- 
cording to the embodiment of Fig. 25, the neuron out- 
put values can be calculated at a high speed by using 
the algorithm of Fig. 5. In the following, the de- 
scription will be made upon the mul t 1 - 1 ayer ed network, 
10 but similar effects can be attained even In the Hop- 
field network. In the embodiment of Fig. 25, the out- 
put value of the first neuron of the s-th layer is 
calculated by raising one word line of the memory cell 
array A to read out the neuron output value of the (s- 
l)th layer, by closing the switches SWl, - - and 
3 SWr to write the neuron output values of the Cs-l)th 

layer in the register 14, and by opening the switches 
fQ SWl, - - and SWr. Next, one word line of the memo- 

= ry cell array B is raised to read the connection 

1^0 weight between the neuron of the (s-l)th layer and the 
fU first neuron of the s-th layer, and the neuron output 

52 values of the (s-l)th layer are read out by the regis- 

n ter 14 so that the output value of the first neuron of 

the s-th layer is calculated by the ar i thme t Ic ci rcui t 
25 12. The calculated results are written in the memory 
cell array A. Simultaneously with this, one word line 
of the memory cell array B is raised to read out the 
connection weight between the neurons of the (s-l)th 
layer and the second neuron of the s-th layer, and the 
30 neuron output values of the (s-l)th layer are read out 
by the register 14 so that the output value of the 
second neuron of the s-th layer is calculated by the 
arithmetic circuit 12. From now on, the output values 
of the neurons of the s-th layer are likewise calcu- 
35 lated. Next, the output values of the neurons of the 




(s+l)th layer are calculated by raising one word line 
of the memory cell array A to read out the previously 
determined neuron output value of the s-th layer, and 
by closing the switches SWl , - - and SWr to write 
5 the neuron output values of the s-th layer in the re- 
gister 14, and the subsequent calculations are pro- 
ceeded like before. As has been described herein- 
before, according to the present embodiment, the writ- 
ing operation and the reading operation can be simul- 
10 taneously carried by providing the register 14 so that 
they can be accomplished at a high speed. 

The description thus far made is directed mainly 
to the method of calculating the "neuron output values 
In according to the present invention, and the necessary 

His connection weights are assumed to be already given. 

Depending upon the subject, the necessary connection 
'"J weights are easily given at the start or have to be 

determined by the so-called "learning**. In the learn- 
s ing for the mul t i - 1 ayered network called the **back 

^720 propagation*', for example, several neuron output 
rO values (or test patterns) of the input layer can be 

\^ prepared in advance to determine the connection 

f5 weights so that the desired neuron output values may 

be obtained in the output layer for the test patterns. 
25 As described in Section 2 of Neural Network Data Pro- 
cessing (published by Sangyo Tosho and edited by 
Hideki Asou) , moreover, there is known the learning 
algorithm for setting' the connection weights such that 
the balanced state of the neuron output values can 
30 take the desired state even in the Hopfield network. 

This learning can be applied to the present invention 
by the following three methods. According to the 
first method, the learning is carried out by using an 
external computer, and the obtained connection weights 
35 are written in the data processing system according to 
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the present invention- This method is advantageous in 
that the learning can be executed by the software so 
that the learning algorithm can be easily changed, but 
is difficult to speed up the learning* According to 
5 the second method, the arithmetic circuit of the sys- 
tem according to the present Invention is given an 
arithmetic function for the learning so that the 
learning is executed on-chip. This method speeds up 
the learning but may be difficult to integrate all the 
10 circuits necessary for the learning over a common 

chip. The third method is an intermediate one between 
the first and second methods, and a portion of the 
arithmetics necessary for the learning is executed by 
3 the system of the present invention whereas the re- 

"'JL5 maining portion of the arithmetics necessary for the 
learning is executed by the external computer. This 
method is advantageous in that it can accelerate the 
Jfi learning speed more than the first method and that the 

5 arithmetic circuit of the system of the present inven- 

jr20 tion can be simply constructed. This third method 
fU will be specifically described in the following. In- 

53 cidentally, the learning method is exemplified by the 

n back propagation method in the mul t i - 1 ayered network. 

In the back propagation method (as will be shortly re- 
25 f erred to as the "BP" method), the connection weights 
are updated according to the following formulas: 

T^ij = T^ij + e djsVis-i ------- (1); 

dj« = (tj - Vj„)g*(Ujm) -------- (2); 

and 

30 das =g'(Uj3)2: 1 (T^*S jdis^i) ------ (3) 

(s = m-1 , - - - , and 2) , 
wherein: 

c : a small positive number; 

tj: a target of the neuron output value Vjm 
35 of the final layer; 



g': a derivative of the nonlinear transfer 
function g; and 
Ujs: a quantity before passage through the 

nonlinear transfer function circuit g in 
the j-th neuron of the s-th layer, as 
will be defined by the following 
formula: 

Uj3 = E i(T^-^4iV,s.x + 0 j3) (4). 

The connection weights may be updated by determining 
the quantities to be updated from the above-specified 
formulas (1) to (4) for every input data for the 
learning and by using the sum of the updated quanti- 
ties of all the "input data for the learning. On the 
other hand, the updating may be carried out by adding 
the following term called the "inertia term" to the * 
f ormul a ( 1 ) : 

H AT^ij* ... (5). 

wherei n : 

jjL : a small positive constant; and 
AT^ij': a corrected quantity of the previous 
updat i ng • 

The updating is continued till the difference between 
the neuron output values of the last layer and the 
target values becomes sufficiently small. 

The learning thus far described can be executed 
by the embodiment shown in. Fig. 23 and the external 
computer, as will be described in the following* The 
description to be made is directed to the case in 
which the updating is executed by summing the updated 
values of all the input data, but similar operations 
are applied to the case in which the connection 
weights are updated for every input data. Inciden- 
tally, the description to be made Is directed to the 
case of the three- 1 ayered network, but similar opera- 
tions are applied to the case of a net work having 
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three or more layers. 

First of all, the input data for all the learn- 
ings and their target values are written in the memo- 
ries A and TG, respectively. Next, the a random num- 
5 ber having a small absolute value is written as the 

initial value of the connection weights in the memory 
B. Moreover, the first input data are read out as the 
neuron output values of the first layer to the arith- 
metic circuit 12b, and the connection weights between 
10 the first and second layers are read out to the arith- 
metic circuit 12b by the memory These values are 
multiplied in parallel by the aforementioned method so 
that the neuron output values of the second layer are 
calculated and written in the memory A. Subsequently, 
^3JS the neuron output values of the third layer are calcu- 
^"^ lated and written in the memory A. The calculations 
"^4 thus far described are executed for all the leaning 
^Ij input data to read out the neuron output values of the 
= individual layers for the individual input data, the 
J2;0 desired values for the individual input data, and the 
?y connection weights to the memory outside of the chip* 
O Next, the quantities for updating the connection 
^ weights are calculated in the external computer, and 

the updated connection weights are written in the 
25 memory B of the system according to the present inven- 
tion- Incidentally, the term g'CUjs) appearing in the 
formulas (2) and (3) may be either calculated from the 
value Uj3 inputted to the nonlinear transfer function 
circuit D, when the neuron output value Vjs is to be 
30 calculated in the system of the present invention, or 
calculated inversely from the value Vjs by the exter- 
nal computer according to the following formula: 

g'(Uj3) = g'(g-MVjs)) - - (6), 

In order to add the inertia term of the formula (5), 
35 on the other hand, the corrected quantities of the 



connection weights may be stored in the memory outside 
of the chip for every updatings so that they may be 
added to the newly determined corrected quantities in 
accordance with the formula (5), 
5 The updatings thus far described can be repeated 

to proceed the learning. In order to know how the 
learning advances, the distances between the neuron 
output values of the last layer for the individual in- 
put data and their desired values can be used as the 

10 measures- These distances can be calculated at a high 
speed by using the embodiment of Fig. 24. As a re- 
sult, it is easily possible to confirm the advance of 
the learning while the 1 earni ng is ^ bei ng accomplished. 
As has been described hereinbefore, according to 

15 the present invention, the calculations of the neuron 
output values for the input data for the learning can 
be executed at a high speed in the system of the pre- 
sent invention. In the present invention, moreover, 
the memories composed of memory arrays are used in the 

20 memories TG, A and B so that all the input data, the 
desired values and the neuron output values of the 
preceding layer can be easily stored and so that the 
numerous bits can be read out in parallel by raising 
the word lines. As a result, the transfers of the 

25 data to the external memory can be executed altogether 
at a high speed. As a result, the learning can be 
proceeded at a high speed. 

If the capacities of the memories are made suffi- 
ciently large in the present embodiment, the number of 

30 neurons can be easily changed according to the ap- 
plication. If, in this case, the neuron number is 
drastically changed, the nonlinear transfer function 
circuit may have to have its dynamic range changed. 
For this necessity, it is possible to switch and use a 

35 plurality of nonlinear transfer function circuits hav- 



ing different characteristics and the reference volt- 
ages of the amplifiers in the nonlinear transfer func- 
tion circuit. In case the neuron numbers are dif- 
ferent for the layers in the mul t i - 1 ayered network, 
5 the nonlinear transfer function circuit may have to 
have its dynamic range changed for the layers. This 
necessity can also be coped with by the similar 
method. 

Incidentally, the description thus far made is 
10 directed to the embodiment In which either the DRAM 
cell of the so-called "one transistor and one capa- 
city" type and the SRAM cells shown in Figs* 16(a) and 
16(b) are mainly used, but other memory cells can 
□ naturally be used in the present invention. Since the 

%5 portion for storing the connection weights need not be 
g frequently rewritten for the data processing, the 

^ kinds of the cells can be changed according to the 

0 contents of the memories by using non-volatile memory 

cells or the DRAM cells or the SRAM cells in the por- 
="20 tion for storing the neuron output values* 
U If the memory cell circuit is highly integrated 

t by using very small memory cells such as the DRAM 

3 cells of the one transistor and one capacitor, some 

memory cells may be sometimes inactive because the 
25 wiring lines used are very small. The neural network 
is advantageous in that its function is hardly in- 
fluenced even if the connection weights are changed 
more or less, but the data processing may be troubled 
in case the memory cells for storing the neuron output 
30 values are inactive. In order to solve this problem, 
the redundant word lines or data lines to be used in 
the ordinary h i gh 1 y- i n t egra t ed semiconductor memory 
can be provided so that defective cells may not be 
used . 

35 In Figs. 14(a), 14(e) nd 14(f) and Fig. 18(d), 



moreover, there are used the bipolar transistors, 
which can be realized by the CMOS, Still moreover, 
the present invention can be practiced by not only the 
bipolar transistors and the MOS transistors but also 
5 other devices. 

Although the foregoing description is directed 
mainly to the mul t i - 1 ayered and Hopfield networks, the 
present invention should not be limited thereto but 
can be applied to the neural network data. process i ng 

10 for the networks of various types. For example, it is 
possible to realize the network in which the updating 
of the neuron output values such as the Boltzman*s 
machine. As described on pp. 27 of Neural Network 
Data Processing (published by Sangyo Tosho and edited 

15 by Hideki Asou) , the Boltzman's machine is featured, 
although the network shape is similar to that of the 
Hopfield network, in that the neuron output value (0 
or 1) is not uniquely determined by another product 
sum of the neuron output values inputted to the neu- 

20 rons and the connection weights but in a probable man- 
ner. The probability P for the neuron output value 
take the value 1 is expressed by P = 1/ (1 + exp(-I/ 
T)), Here, letter I designates the product sum of the 
nol inputted to the neurons and the connection wei- 

25 ghts, and letter T designates a parameter called the 
temperature. The Boltzman's machine described above 
can be easily realized according to the present inven- 
tion. For example, the reference voltage VRx of the , 
nonlinear transfer function circuit D, as shown in 

30 Fig. 18(d), is not set to the steady value but may be 
changed with time within the fluctuating range of the 
product/sum output line NO, Then, the neuron output 
values can be determined according to the probability. 
The effects obtained by changing the changing rate are 

35 similar to those obtained by changing the temperature 



T. 

If the capacity of the memories is sufficient, as 
is apparent from the comparisons between Fig. 8(a) and 
Figs. 20(a) and 20(b), various types of network can be 
5 realized by a common system merely by changing the ad- 
dresses of the memory cells for storing the neuron 
output values and the connection weights. Thus, the 
present invention has a highly wide applicability. 
Although the description thus far made is di- 

10 rected to the applications to the neural network data 
processing, the present invention should not be limit- 
ed thereto but can naturally realize such a system in 
a high degree of integration as is used for the data 
processing by connecting a number of processing ele- 

15 ments having similar processing functions in the form 
of a network. 

In the embodiments thus far described, the de- 
scription is directed mainly to the structure in which 
the arithmetic circuit performs the analog arithme- 

20 tics. The analog arithmetic circuit is advantageous 

in that it has a high speed and a small circuit scale. 
Despite of this description, however, the present in- 
vention should not be limited thereto but can be used 
in a digital arithmetic circuit without departing from 

25 the gist thereof. In this case, the calculations can 
be highly accurately executed by the digital arith- 
metic ci rcui t . 

As has been described hereinbefore, according to 
the present invention, the system for executing the 

30 data processing by combining the memories and the 
arithmetic circuits and by performing the parallel 
arithmetics with the arithmetic circuits like the 
parallel distributed processing system such as the 
neural network, in which a number of arithmetic units 

35 for relatively simple arithmetics are connected in the 
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network form, can be realized with a hi^h degree of 
integration without sacrificing the speed. 

It is further understood by those in the art that 
the foregoing description is preferred embodiment of 
5 the disclosed device and that various changes and 
modifications may be made in the invention without 
departing from the spirit and scope thereof. 



